Mobile application interface device for vehicle navigation assistance

ABSTRACT

An interface device connects to a mobile application running on a mobile device to provide vehicle navigation assistance. The interface device includes a microphone, a location determining component, a display, a plurality of indicator lights, a short-range communications transceiver, and a controller. The controller is configured to: receive a spoken instruction or query from a user via the microphone; send data associated with the spoken instruction or query to the mobile application running on the mobile device via the short-range communications transceiver; receive navigation data based on the spoken instruction or query from the mobile application running on the mobile device via the short-range communications transceiver; determine a current position based on one or more signals received via the location determining component; and provide at least one of a symbolic output or a textual output via the display based on the navigation data and the current position.

RELATED APPLICATIONS

The present application claims the benefit under 35 U.S.C. § 119(e) ofU.S. Provisional Application Ser. No. 62/486,653, entitled “VehicleAssistant Interface,” filed on Apr. 18, 2017, Provisional ApplicationSer. No. 62/563,962, entitled “Vehicle Assistant Interface,” filed onSeptember 27, and Provisional Application Ser. No. 62/596,990, entitled“Vehicle Assistant Interface,” filed on Dec. 11, 2017. Theabove-referenced Provisional Applications are herein incorporated byreference in their entirety.

BACKGROUND

Drivers rely on various technologies to assist with vehicle navigationand media control. For example, many vehicles come equipped with aninfotainment console that includes a navigation system and/or mediacontrol interface. Drivers may additionally or alternatively rely ontheir mobile devices (e.g., smartphones) to access navigation or mediaapplications, which may be streamed through a vehicle infotainmentand/or audio system, or presented directly via the mobile device displayand/or speakers.

Using a driver's mobile device for navigation or media applications canbe cumbersome and possibly dangerous when a vehicle does not include aninfotainment console or when the infotainment console cannot be linkedto the driver's mobile device. For example, the mobile device mayreceive calls while navigating, thereby interrupting route guidanceand/or creating a distraction for the driver. Also, at times thedriver's mobile device may be located in a difficult to view area withinthe vehicle (e.g., within a cup holder). These are some examples of thedifficulties that can be encountered when a driver uses his/her mobiledevice for navigation or media applications while driving.

SUMMARY

A device for interfacing with a mobile application running on a mobiledevice to provide vehicle navigation assistance is disclosed. Inembodiments, the device includes a microphone, a location determiningcomponent, a display, a plurality of indicator lights, a short-rangecommunications transceiver, and a controller. The short-rangecommunications transceiver is configured to send and receivecommunications to and from the mobile device. The controller iscommunicatively coupled with the microphone, the location determiningcomponent, the display, the plurality of indicator lights, and theshort-range communications transceiver. The controller is configured toreceive a spoken instruction or query from a user via the microphone andconfigured to send data associated with the spoken instruction or queryto the mobile application running on the mobile device via theshort-range communications transceiver. The controller is furtherconfigured to receive navigation data based on the spoken instruction orquery from the mobile application running on the mobile device via theshort-range communications transceiver and configured to determine acurrent position based on one or more signals received via the locationdetermining component. The controller is configured to provide at leastone of a symbolic output or a textual output via the display based onthe navigation data and the current position. The controller can also beconfigured to cause the plurality of indicator lights to display anillumination pattern based on the navigation data, current position,and/or the spoken instruction or query received via the microphone fromthe user.

A method of providing vehicle navigation assistance is also disclosed.In implementations, the method includes: running a mobile applicationfor vehicle navigation assistance on a mobile device; pairing a mobileapplication interface device with the mobile device, the mobileapplication interface device having a microphone, a location determiningcomponent, a display, a plurality of indicator lights, and a short-rangecommunications transceiver operable to send and receive communicationsto and from the mobile device; receiving a spoken instruction or queryfrom a user via the microphone of the mobile application interfacedevice; sending data associated with the spoken instruction or query tothe mobile application running on the mobile device via the short-rangecommunications transceiver of the mobile application interface device;receiving navigation data based on the spoken instruction or query fromthe mobile application running on the mobile device via the short-rangecommunications transceiver of the mobile application interface device;determining a current position based on one or more signals received viathe location determining component of the mobile application interfacedevice; and providing at least one of a symbolic output or a textualoutput via the display of the mobile application interface device basedon the navigation data and the current position, wherein the at leastone of the symbolic output or the textual output is associated with atleast one of a next maneuver for the user to follow or a distance to adestination. In some implementations, the method further includescausing the plurality of indicator lights to display an illuminationpattern based on the navigation data, current position, and/or thespoken instruction or query received via the microphone from the user.

This Summary is provided solely as an introduction to subject matterthat is fully described in the Detailed Description and Drawings. TheSummary should not be considered to describe essential features nor beused to determine the scope of the Claims. Moreover, it is to beunderstood that both the foregoing Summary and the following DetailedDescription are example and explanatory only and are not necessarilyrestrictive of the subject matter claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The detailed description is described with reference to the accompanyingfigures. The use of the same reference numbers in different instances inthe description and the figures may indicate similar or identical items.Various embodiments or examples (“examples”) of the present disclosureare disclosed in the following detailed description and the accompanyingdrawings. The drawings are not necessarily to scale. In general,operations of disclosed processes may be performed in an arbitraryorder, unless otherwise provided in the claims.

FIG. 1A is a perspective front view of a mobile application interfacedevice for vehicle navigation assistance, in accordance with an exampleembodiment of the present disclosure.

FIG. 1B is a perspective rear view of a mobile application interfacedevice for vehicle navigation assistance, in accordance with an exampleembodiment of the present disclosure.

FIG. 1C is a block diagram illustrating various components of a mobileapplication interface device, such as the mobile application interfacedevice of FIGS. 1A and 1B, in accordance with an example embodiment ofthe present disclosure.

FIG. 2A is a block diagram illustrating a system that can employ amobile application interface device, such as the mobile applicationinterface device of FIGS. 1A through 1C, in accordance with an exampleembodiment of the present disclosure.

FIG. 2B is a block diagram illustrating a mobile device that can beconfigured to communicate with a mobile application interface device,such as the mobile application interface device of FIGS. 1A through 1C,in accordance with an example embodiment of the present disclosure.

FIG. 2C is a block diagram illustrating a vehicle audio system that canbe configured to communicate with a mobile application interface device,such as the mobile application interface device of FIGS. 1A through 1C,and/or a mobile device, such as the mobile device of FIG. 2B, inaccordance with an example embodiment of the present disclosure.

FIG. 3 shows an example output from a mobile application interfacedevice, such as the mobile application interface device of FIGS. 1Athrough 1C.

FIG. 4 shows an example output from a mobile application interfacedevice, such as the mobile application interface device of FIGS. 1Athrough 1C.

FIG. 5A shows an example output from a mobile application interfacedevice, such as the mobile application interface device of FIGS. 1Athrough 1C.

FIG. 5B shows an example output from a mobile application interfacedevice, such as the mobile application interface device of FIGS. 1Athrough 1C.

FIG. 5C shows an example output from a mobile application interfacedevice, such as the mobile application interface device of FIGS. 1Athrough 1C.

FIG. 5D shows an example output from a mobile application interfacedevice, such as the mobile application interface device of FIGS. 1Athrough 1C.

FIG. 5E shows an example output from a mobile application interfacedevice, such as the mobile application interface device of FIGS. 1Athrough 1C.

FIG. 5F shows an example output from a mobile application interfacedevice, such as the mobile application interface device of FIGS. 1Athrough 1C.

FIG. 5G shows an example output from a mobile application interfacedevice, such as the mobile application interface device of FIGS. 1Athrough 1C.

FIG. 5H shows an example output from a mobile application interfacedevice, such as the mobile application interface device of FIGS. 1Athrough 1C.

FIG. 6 shows an example output from a mobile application interfacedevice, such as the mobile application interface device of FIGS. 1Athrough 1C.

FIG. 7 shows an example output from a mobile application interfacedevice, such as the mobile application interface device of FIGS. 1Athrough 1C.

FIG. 8A shows an example output from a mobile application interfacedevice, such as the mobile application interface device of FIGS. 1Athrough 1C.

FIG. 8B shows an example output from a mobile application interfacedevice, such as the mobile application interface device of FIGS. 1Athrough 1C.

FIG. 9A shows an example output from a mobile application interfacedevice, such as the mobile application interface device of FIGS. 1Athrough 1C.

FIG. 9B shows an example output from a mobile application interfacedevice, such as the mobile application interface device of FIGS. 1Athrough 1C.

FIG. 9C shows an example output from a mobile application interfacedevice, such as the mobile application interface device of FIGS. 1Athrough 1C.

FIG. 9D shows an example output from a mobile application interfacedevice, such as the mobile application interface device of FIGS. 1Athrough 1C.

FIG. 9E shows an example output from a mobile application interfacedevice, such as the mobile application interface device of FIGS. 1Athrough 1C.

FIG. 10 shows audio output/alert priorities for a mobile applicationinterface device, such as the mobile application interface device ofFIGS. 1A through 1C, in accordance with an example embodiment of thepresent disclosure.

FIG. 11 is a flow diagram showing an example process that employs amobile application interface device, such as the mobile applicationinterface device of FIGS. 1A through 1C.

FIG. 12A shows an example output from a mobile application interfacedevice, such as the mobile application interface device of FIGS. 1Athrough 1C.

FIG. 12B shows an example output from a mobile application interfacedevice, such as the mobile application interface device of FIGS. 1Athrough 1C.

FIG. 13 shows display output/alert priorities for a mobile applicationinterface device, such as the mobile application interface device ofFIGS. 1A through 1C, in accordance with an example embodiment of thepresent disclosure.

FIG. 14 is a flow diagram showing an example process that employs amobile application interface device, such as the mobile applicationinterface device of FIGS. 1A through 1C.

FIG. 15A is a flow diagram showing an example process that employs amobile application interface device, such as the mobile applicationinterface device of FIGS. 1A through 1C.

FIG. 15B is a flow diagram showing an example process that employs amobile application interface device, such as the mobile applicationinterface device of FIGS. 1A through 1C.

FIG. 15C is a flow diagram showing an example process that employs amobile application interface device, such as the mobile applicationinterface device of FIGS. 1A through 1C.

FIG. 15D is a flow diagram showing an example process that employs amobile application interface device, such as the mobile applicationinterface device of FIGS. 1A through 1C.

FIG. 16A is a flow diagram showing an example process that employs amobile application interface device, such as the mobile applicationinterface device of FIGS. 1A through 1C.

FIG. 16B is a flow diagram showing an example process that employs amobile application interface device, such as the mobile applicationinterface device of FIGS. 1A through 1C.

FIG. 17A shows an example output from a mobile application interfacedevice, such as the mobile application interface device of FIGS. 1Athrough 1C.

FIG. 17B shows an example output from a mobile application interfacedevice, such as the mobile application interface device of FIGS. 1Athrough 1C.

FIG. 18A is a flow diagram showing an example process that employs amobile application interface device, such as the mobile applicationinterface device of FIGS. 1A through 1C.

FIG. 18B is a flow diagram showing an example process that employs amobile application interface device, such as the mobile applicationinterface device of FIGS. 1A through 1C.

FIG. 19 is a flow diagram showing an example process that employs amobile application interface device, such as the mobile applicationinterface device of FIGS. 1A through 1C.

FIG. 20A is a flow diagram showing an example process that employs amobile application interface device, such as the mobile applicationinterface device of FIGS. 1A through 1C.

FIG. 20B is a flow diagram showing an example process that employs amobile application interface device, such as the mobile applicationinterface device of FIGS. 1A through 1C.

FIG. 21A is a flow diagram showing an example process that employs amobile application interface device, such as the mobile applicationinterface device of FIGS. 1A through 1C.

FIG. 21B is a flow diagram showing an example process that employs amobile application interface device, such as the mobile applicationinterface device of FIGS. 1A through 1C.

FIG. 22 is a flow diagram showing an example process that employs amobile application interface device, such as the mobile applicationinterface device of FIGS. 1A through 1C.

FIG. 23 is a flow diagram showing an example process that employs amobile application interface device, such as the mobile applicationinterface device of FIGS. 1A through 1C.

FIG. 24 is a flow diagram showing an example process that employs amobile application interface device, such as the mobile applicationinterface device of FIGS. 1A through 1C.

FIG. 25A is a flow diagram showing an example process that employs amobile application interface device, such as the mobile applicationinterface device of FIGS. 1A through 1C.

FIG. 25B is a map view showing a user approaching a destination whileemploying a mobile application interface device, such as the mobileapplication interface device of FIGS. 1A through 1C.

FIG. 25C is a map view showing a user approaching a destination whileemploying a mobile application interface device, such as the mobileapplication interface device of FIGS. 1A through 1C.

FIG. 26A is a flow diagram showing an example process that employs amobile application interface device, such as the mobile applicationinterface device of FIGS. 1A through 1C.

FIG. 26B is a flow diagram showing an example process that employs amobile application interface device, such as the mobile applicationinterface device of FIGS. 1A through 1C.

FIG. 27A is a flow diagram showing an example process that employs amobile application interface device, such as the mobile applicationinterface device of FIGS. 1A through 1C.

FIG. 27B shows an example output from a mobile application interfacedevice, such as the mobile application interface device of FIGS. 1Athrough 1C.

FIG. 28A is a flow diagram showing an example process that employs amobile application interface device, such as the mobile applicationinterface device of FIGS. 1A through 1C.

FIG. 28B shows an example output from a mobile application interfacedevice, such as the mobile application interface device of FIGS. 1Athrough 1C.

FIG. 29 is a flow diagram showing an example process that employs amobile application interface device, such as the mobile applicationinterface device of FIGS. 1A through 1C.

FIG. 30A is a flow diagram showing an example process that employs amobile application interface device, such as the mobile applicationinterface device of FIGS. 1A through 1C.

FIG. 30B is a flow diagram showing an example process that employs amobile application interface device, such as the mobile applicationinterface device of FIGS. 1A through 1C.

FIG. 31A shows example display output symbols for a mobile applicationinterface device, such as the mobile application interface device ofFIGS. 1A through 1C.

FIG. 31B shows example display output symbols for a mobile applicationinterface device, such as the mobile application interface device ofFIGS. 1A through 1C.

FIG. 31C shows example display output symbols for a mobile applicationinterface device, such as the mobile application interface device ofFIGS. 1A through 1C.

FIG. 31D shows example display output symbols for a mobile applicationinterface device, such as the mobile application interface device ofFIGS. 1A through 1C.

FIG. 31E shows example display output symbols for a mobile applicationinterface device, such as the mobile application interface device ofFIGS. 1A through 1C.

FIG. 32 is a block diagram illustrating a system that employs a mobileapplication interface device, such as the mobile application interfacedevice of FIGS. 1A through 1C, in accordance with an example embodimentof the present disclosure.

FIG. 33 is a block diagram illustrating a system that employs a mobileapplication interface device, such as the mobile application interfacedevice of FIGS. 1A through 1C, in accordance with an example embodimentof the present disclosure.

FIG. 34 is a block diagram illustrating a system configuration for amobile application interface device, such as the mobile applicationinterface device of FIGS. 1A through 1C, in accordance with an exampleembodiment of the present disclosure.

FIG. 35 is a block diagram illustrating a system configuration for amobile application interface device, such as the mobile applicationinterface device of FIGS. 1A through 1C, in accordance with an exampleembodiment of the present disclosure.

FIG. 36 is a block diagram illustrating a text-to-speech (TTS) guidancesystem configuration for a mobile application interface device, such asthe mobile application interface device of FIGS. 1A through 1C, inaccordance with an example embodiment of the present disclosure.

FIG. 37 is a block diagram illustrating a system configuration for amobile application interface device, such as the mobile applicationinterface device of FIGS. 1A through 1C, wherein the mobile applicationinterface device facilitates communication with network providednavigation services, in accordance with an example embodiment of thepresent disclosure.

FIG. 38 is a block diagram illustrating a system configuration for amobile application interface device, such as the mobile applicationinterface device of FIGS. 1A through 1C, wherein the mobile applicationinterface device facilitates communication with network provided voiceservices, in accordance with an example embodiment of the presentdisclosure.

FIG. 39 is a block diagram illustrating a system configuration for amobile application interface device, such as the mobile applicationinterface device of FIGS. 1A through 1C, wherein the mobile applicationinterface device facilitates communication with network providednavigation and voice services, in accordance with an example embodimentof the present disclosure.

FIG. 40 is a block diagram illustrating a system configuration for amobile application interface device, such as the mobile applicationinterface device of FIGS. 1A through 1C, wherein the mobile applicationinterface device facilitates communication with network providednavigation and voice services, in accordance with an example embodimentof the present disclosure.

FIG. 41A is a block diagram illustrating a system configuration for amobile application interface device, such as the mobile applicationinterface device of FIGS. 1A through 1C, wherein the mobile applicationinterface device facilitates communication with network providednavigation and voice services, in accordance with an example embodimentof the present disclosure.

FIG. 41B is a block diagram illustrating a system configuration for amobile application interface device, such as the mobile applicationinterface device of FIGS. 1A through 1C, wherein the mobile applicationinterface device facilitates communication with network providednavigation and voice services, in accordance with an example embodimentof the present disclosure.

FIG. 42A is a block diagram illustrating a system configuration for amobile application interface device, such as the mobile applicationinterface device of FIGS. 1A through 1C, and a home applicationinterface device, wherein the mobile application interface device andthe home application interface device a both communicatively coupled viaa network that facilitates communication with navigation and voiceservices, in accordance with an example embodiment of the presentdisclosure.

FIG. 42B is a block diagram illustrating a system configuration for amobile application interface device, such as the mobile applicationinterface device of FIGS. 1A through 1C, and a home applicationinterface device, wherein the mobile application interface device andthe home application interface device a both communicatively coupled viaa network that facilitates communication with navigation and voiceservices, in accordance with an example embodiment of the presentdisclosure.

DETAILED DESCRIPTION

Drivers may rely on their mobile devices (e.g., smartphones, tablets,media players, or the like) to access navigation or media applicationsthat can be streamed through a vehicle infotainment and/or audio system,or presented directly via the mobile device display and/or speakers. Forexample, a driver may connect a mobile device to a vehicle audio systemto take advantage of mobile assistant applications (e.g., artificialintelligence applications and/or voice services), such as GOOGLEASSISTANT, SIRI, CORTANA, ALEXA, and the like. However, using a driver'smobile device for navigation or media applications can be cumbersome andpossibly dangerous when a vehicle does not include an infotainmentconsole or when the infotainment console cannot be linked to thedriver's mobile device. For example, the mobile device may receive callswhile navigating, thereby interrupting route guidance and/or creating adistraction for the driver. Also, at times the driver's mobile devicemay be located in a difficult to view area within the vehicle (e.g.,within a cup holder). These are some examples of the difficulties thatcan be encountered when a driver uses a mobile device for navigation ormedia applications while driving.

A mobile application interface device is described herein that movesintelligent assistant systems, like SIRI, GOOGLE VOICE, AMAZON VOICESERVICES (AVS or ALEXA), and so forth, from home implementations to avehicle environment, thereby providing drivers with an intelligentdigital assistant while on the road. The mobile application interfacedevice (sometimes referred to herein as the “device”) can provide theconnected functionality many users desire without requiring users todirectly interact with their mobile devices (e.g., without requiring auser to take a mobile phone out of the user's pocket). In embodiments,the device may include a housing, a controller, a short-rangecommunications transceiver (e.g., Bluetooth or NFC transceiver) forcommunicating with a mobile device and possibly with a vehicle audiosystem, an audio output device (e.g., speaker or audio outputinterface), a display, a location determining component (e.g., GPSreceiver or the like), and a mount (e.g., a windshield or dash mount)for securing the device within a vehicle (e.g., to the vehicle'swindshield, dashboard, or any other location that can be easily viewedby a driver). In embodiments, the device may be configured to determineits geographical position using the location determining component. Inother embodiments, the device may utilize positioning informationprovided by the mobile device and/or the user's vehicle. Similarly, invarious embodiments, the device may lack speakers, buttons, displays,and other system elements and instead rely on complimentary elementsprovided by the mobile device and/or vehicle audio system.

The device can be configured to provide various functionalities,including, but not limited to, streaming entertainment (e.g., music,podcasts, internet radio, travel games, audio books, news, etc.),navigation and searching functionalities (e.g., GARMIN searchfunctionality (standard point of interest (POI), address, and/orcategory searching), AMAZON search functionality, YELP, audio and/orvisual navigation instructions (e.g., turn-by-turn, lane guidance,natural guidance, traffic reports, advanced driver awareness and safety(ADAS) alerts/warnings, etc.), dash cam capability/control (e.g.,picture and video recording, time-lapse recording, etc.), mobile devicenotifications (e.g., calendar notifications, reminders/alarms,messaging, emails, etc.), hands-free calling, digital assistance (e.g.,reminders, lists, notes, calendar updates, message transmission orretrieval), smart home automation or security control, assistanceordering goods or services, making reservations, location based queries(e.g., “Where am I?”, weather, traffic, speed limit updates for currentlocation/destination), and so forth.

FIGS. 1A through 1C illustrate embodiments of a device 100 forinterfacing with a mobile application running on a mobile device (e.g.,as illustrated in FIG. 2A—mobile application 218 running on mobiledevice 202). As shown in FIG. 1C, the device 100 includes a controller124 communicatively coupled to one or more input devices 138 and one ormore output devices 140. In embodiments, an input device 138 caninclude, but is not limited to, a microphone 110, a camera 122, anelectromechanical input device 108 (e.g., one or more buttons, switches,or toggles), and/or a touch-sensitive input device 142 (e.g., a touchpad, touch panel, or the like). In embodiments, an output device 140 caninclude, but is not limited to, a speaker 120, a display 104, one ormore indicator lights 106, and/or an audio output interface 144 (e.g., aline out audio port or connector). The device 100 can further include ashort-range communications transceiver 132 (e.g., a Bluetoothtransceiver, near field communications (NFC) transceiver, WiFitransceiver, or the like). For example, as further described herein, thedevice 100 can be configured to send and receive communications to andfrom a mobile device (e.g., mobile device 202). In embodiments, thedevice 100 further includes a location determining component 134, suchas, but not limited to, a Global Navigation Satellite System (GNSS)receiver (e.g., GPS receiver, GLONASS receiver, Galileo receiver, Beidoureceiver, multi-protocol receiver, software defined GNSS receiver, or acombination thereof, or the like. In some embodiments, the device 100may further include a magnetometer 136 for sensing a direction, heading,or bearing of the device 100 (or vehicle in which the device 100 isdeployed). The device 100 may also include additional sensors (e.g.,barometer, thermometer, speed sensor, proximity sensor, inertial sensor(e.g., accelerometer or gyroscope), any combination thereof, and soforth).

As shown in FIGS. 1A and 1B, the device 100 has a housing 102 thatcontains or otherwise provides supportive structure for other componentsmaking up the device. In some embodiments, the housing 102 iscylindrical, for example, as shown in FIGS. 1A and 1B. The housing maybe constructed as a rectangular prism or any other shape; however, thecylindrical design can help reduce the windshield area blocked orobstructed by the housing while still allowing visual and audiointeraction with the device. The overall size of the device 100 and/orits housing 102 may be determined by the internal components of thedevice 100. In some embodiments, the housing 102 is as small as possiblewhile still containing or providing support for all other components ofthe device 100. In embodiments, the housing 102 is coupled to a mount112 for securing the device 100 to a portion (e.g., windshield ordashboard) of a vehicle. The mount 112 can include a first end 116configured to pivotally couple the mount 112 to the housing 102 (e.g.,in a ball and socket configuration, a hinged configuration, or thelike). The mount 112 can also include a second end 114 that defines abase with a suction, adhesive, or high friction surface 118 configuredto secure the second end 114 to a portion (e.g., windshield ordashboard) of the vehicle, where the driver is then enabled to view thedevice 100 comfortably while operating the vehicle.

In an embodiment, the display 104 is disposed at one end or facet (e.g.,front-facing surface) of the device 100 defined by the housing 102. Thedisplay 104 may be a light-emitting diode (LED) display, organiclight-emitting diode (OLED) display, liquid crystal display (LCD), orthe like. For example, in an embodiment, the display 104 is 0.84″ OLEDmono display. The device 100 may further include a plurality ofindicator lights 106 (e.g., LEDs) proximate to the display 104. Forexample, the indicator lights 106 may be disposed about at least aportion of the display 104. In an embodiment, the indicator lights 106form segments of a lighted ring (or portion of a ring) that completelyor partially surrounds the display 104. As further described herein, theindictor lights 106 can be used in conjunction with or instead of thedisplay 104 to provide visual prompts (e.g., connection information,speed warnings, distance/proximity indications, traffic alerts, forwardcollision warnings, lane departure warnings, direction indications,combinations thereof, and so forth). In embodiments, the display 104 mayhave a discrete appearance and may blend in with the front of the device100 until information is provided via the display 104. For example, thedisplay 104 and its surrounding features may appear black until thedisplay 104 is active.

In some embodiments, the indicator lights 106 (e.g., LED segmentsforming a light ring ring) are configured to present maneuver andbearing information. For example, the indicator lights can form a lightring around the perimeter of a surface of the housing (e.g., around thedisplay 104) that may be used for providing navigation information andinstructions to the user. The controller 124 can be configured topresent a light pattern via the indicator lights 106 to point the usertowards the first road in a navigation route if they are in a parkinglot and point the user back towards their route if they leave thenavigation route (e.g., if the device cannot match the user's positionto the active route or is recalculating the route). In some embodiments,the indicator lights 106 form a light ring that can dynamically fill up(e.g., from the bottom of the ring towards the top of the ring, leftside to right side, or vice versa) as the user nears a turn maneuver toindicate and communicate a distance from a current geographic locationto said turn maneuver. In some embodiments, the controller 124 can alsobe configured to provide a similar dynamic light pattern (e.g.,gradually filling in portions of the light ring) to indicate proximityto a destination (e.g., the light ring can fill in from the bottom ofthe ring towards the top of the ring, left side to right side, or viceversa, as the user nears the destination).

In FIG. 1A, the device 100 is shown to include at least one microphone110 positioned to receive spoken instructions or queries from a driver.For example, the microphone 110 is disposed proximate to the display104. As shown in FIG. 1B, the speaker 120 may be disposed at an end orfacet of the device 100 that is opposite the display 104 (e.g., at arear-facing surface of the device 100). In some embodiments, the speaker120 is configured to output audible audio signals; however, the device100 may alternatively or additionally rely on a vehicle audio systemand/or mobile device 200 for audio output capabilities.

In FIG. 1B, the device 100 is further shown to include a camera 122(e.g., a dash cam) disposed proximate to the speaker 120. In someembodiments, the device 100 may include an accelerometer within thehousing 102 that enables the device 100 to determine if the camera 122is orientated such that it points straight ahead. The controller 124 maybe configured to inform the driver (e.g., via the display or an audiooutput) to re-orient (re-aim) the camera 122 so features relying on data(images) received from the camera 122, such as lane departure warningand forward collision warning, can work properly. In some embodiments,the device 100 does not include a camera 122, and instead the speaker120 may occupy an entirety of the rear-facing surface of the device 100.

In the embodiments illustrated in FIGS. 1A and 1B, the device alsoincludes one or more buttons 108 (e.g., a mic mute button and an actionbutton). The foregoing are examples of input devices 138 and outputdevices 140 that the device can include in some embodiments, but thedevice 100 can include other combinations of input devices 138 andoutput devices 140, such as those shown in FIG. 1C or any combinationthereof.

The controller 124 is communicatively coupled with some or all of thecomponents of the device 100. For example, the controller 124 can becommunicatively coupled with the input device(s) 138, the outputdevice(s) 140, short-range communications transceiver 132, locationdetermining component 134, and any additional sensors or othercomponents of the device 100. The controller 124 has a processor 126included with or in the controller 124 to control the components andfunctions of the device 100 described herein using software, firmware,hardware (e.g., fixed logic circuitry), or a combination thereof. Theterms “controller,” “functionality,” “service,” and “logic” as usedherein generally represent software, firmware, hardware, or acombination of software, firmware, or hardware in conjunction withcontrolling the device 100. As shown in FIG. 1C, the controller 124 caninclude a processor 126, a memory 128, and a communications interface130.

The processor 126 provides processing functionality for at least thecontroller 124 and can include any number of processors,micro-controllers, circuitry, field programmable gate array (FPGA) orother processing systems, and resident or external memory for storingdata, executable code, and other information accessed or generated bythe controller 124. The processor 126 can execute one or more softwareprograms embodied in a non-transitory computer readable medium thatimplement techniques described herein. The processor 126 is not limitedby the materials from which it is formed or the processing mechanismsemployed therein and, as such, can be implemented via semiconductor(s)and/or transistors (e.g., using electronic integrated circuit (IC)components), and so forth.

The memory 128 can be a tangible, computer-readable storage medium thatprovides storage functionality to store various data and or program codeassociated with operation of the controller 124, such as softwareprograms and/or code segments, or other data to instruct the processor126, and possibly other components of the device 100/controller 124, toperform the functionality described herein. The memory 128 can storedata, such as a program of instructions for operating the device 100(including its components), and so forth. It should be noted that whilea single memory 128 is described, a wide variety of types andcombinations of memory (e.g., tangible, non-transitory memory) can beemployed. The memory 128 can be integral with the processor 126, cancomprise stand-alone memory, or can be a combination of both. Someexamples of the memory 128 can include removable and non-removablememory components, such as random-access memory (RAM), read-only memory(ROM), flash memory (e.g., a secure digital (SD) memory card, a mini-SDmemory card, and/or a micro-SD memory card), magnetic memory, opticalmemory, universal serial bus (USB) memory devices, hard disk memory,external memory, and so forth. In embodiments, the device 100 and/or thememory 128 can include removable integrated circuit card (ICC) memory,such as memory provided by a subscriber identity module (SIM) card, auniversal subscriber identity module (USIM) card, a universal integratedcircuit card (UICC), and so on.

The communications interface 130 can be operatively configured tocommunicate with components of the device 100. For example, thecommunications interface 130 can be configured to transmit data forstorage in the device 100, retrieve data from storage in the device 100,and so forth. The communications interface 130 can also becommunicatively coupled with the processor 126 to facilitate datatransfer between components of the device 100 and the processor 126(e.g., for communicating inputs to the processor 126 received from adevice communicatively coupled with the controller 124, including, butnot limited to, data received from the location determining component134, any input device 138, and/or any other component of the device100). It should be noted that while the communications interface 130 isdescribed as a component of controller 124, one or more components ofthe communications interface 130 can be implemented as components of thedevice 100 or components communicatively coupled to the device 100 via awired and/or wireless connection. For example, the device 100 and/or thecontroller 124 includes the short-range communications transceiver 132(or in some embodiments, a transmitter and a receiver) for sending andreceiving communications to and from a mobile device and/or vehicleaudio system.

FIG. 2A illustrates an embodiment of a system 200 for vehicle navigationassistance that employs the device 100. The system 200 includes thedevice 100 and a mobile device 202 running a mobile application 218 forvehicle navigation assistance, where the device 100 is configured tosend and receive data to and from the mobile application 218 running onthe mobile device 202. The mobile application 218 is configured tofacilitate communications between the device 100 and a first server 206for voice service processing (e.g., AMAZON Voice Services, or the like)and a second server 208 for navigation/mapping service (e.g., GARMINNavigation Services, or the like). For example, the mobile application218 can send and receive mobile data communications to and from theservers 206 and 208 via a cellular transceiver 222 of the mobile device202. In embodiments, the servers 206 and 208 can also be configured tocommunicate with one another. In other embodiments, the mobileapplication 218 includes or is connected to a navigation skill/client onthe mobile device 202 that facilitates communications between the firstserver 206 and the second server 208. For example, the mobileapplication 218 can be configured to send data to the first server 206for voice service processing based on spoken instructions/queriesreceived via the device 100, and in turn, the first server 206 caneither communicate an instruction (e.g., navigation command data) to thesecond server 208 in a format acceptable to the second server 208, orthe first server 206 can send the instruction to the mobile application218 that then communicates the instruction to the second server 208. Inresponse, the second server 208 can send navigation data (e.g., routeguidance/information, point of interest information, etc.) to the mobileapplication 218 and/or to the first server 206.

The device 100 and/or the mobile device 202 may also be communicativelycoupled with a vehicle audio system 204 according to one or moreconfigurations. For example, in one configuration, the mobile device 202is configured to transmit audio outputs to the vehicle audio system 204(e.g., via a Bluetooth connection (e.g., A2DP connection), NFCconnection, Wi-Fi connection, or the like). In another configuration,the mobile device 202 is configured to transmit audio outputs to thedevice 100 that then transmits the audio outputs to the vehicle audiosystem 204 (e.g., via the audio output interface 144 or the short-rangecommunications transceiver 132 of the device 100). In anotherconfiguration (e.g., in vehicles that have hands-free calling but lackA2DP capability), the mobile device 202 is configured to handle callsover a Bluetooth connection with the vehicle audio system 204 and isconfigured to transmit other audio outputs (e.g., guidance/information)to the device 100 that can then transmit the audio outputs to thevehicle audio system 204 via the audio output interface 144 of thedevice 100 (e.g., through an auxiliary input of the vehicle audio system204 or the like). In other configurations, audio may be rendered throughthe device 100 and/or mobile device 202 speakers.

An embodiment of the mobile device 202 is shown in FIG. 2B. The mobiledevice 202 may be a smartphone, media player, tablet, smartwatch, or thelike. In embodiments, the mobile device 202 includes a controller 210communicatively coupled to one or more input devices 226 and one or moreoutput devices 244. In embodiments, an input device 226 can include, butis not limited to, an electromechanical input device 228 (e.g., one ormore buttons, keypad, switches, or toggles), a touch-sensitive inputdevice 230 (e.g., a touch pad, touch panel, or the like), a microphone240, and/or a camera 242. In embodiments, an output device 244 caninclude, but is not limited to, a speaker 246, a display 248, one ormore indicator lights 250, and/or an audio output interface 252 (e.g., aline out audio port or connector). The mobile device 202 can include ashort-range communications transceiver 220 (e.g., a Bluetoothtransceiver, near field communications (NFC) transceiver, WiFitransceiver, or the like). For example, as described herein, the mobiledevice 202 can be configured to communicate with the device 100 and/orthe vehicle audio system 204 via the short-range communicationstransceiver 220. The mobile device 202 can also include a cellulartransceiver 222 (e.g., 2G, 3G, 4G, and/or LTE transceiver or the like)for sending and receiving mobile data and handling calls. For example,the mobile device 202 can be configured to communicate with the servers206 and 208 for voice and navigation services via the cellulartransceiver 222. In embodiments, the mobile device 202 further includesa location determining component 224, such as, but not limited to, aGlobal Navigation Satellite System (GNSS) receiver (e.g., GPS receiver,GLONASS receiver, Galileo receiver, Beidou receiver, multi-protocolreceiver, software defined GNSS receiver, or a combination thereof, orthe like. For example, the mobile device 202 can be used instead of orin addition to the device 100 to determine a current location (e.g.,vehicle position) for route guidance and/or information (e.g. point ofinterest searching, mapping, traffic updates, etc.).

The controller 210 is communicatively coupled with some or all of thecomponents of the mobile device 202. For example, the controller 210 canbe communicatively coupled with the input device(s) 226, the outputdevice(s) 244, short-range communications transceiver 220, cellulartransceiver 222, and any sensors or other components (e.g., locationdetermining component 224) of the mobile device 202. The controller 210has a processor 212 included with or in the controller 210 to controlthe components and functions of the mobile device 202 described hereinusing software, firmware, hardware (e.g., fixed logic circuitry), or acombination thereof. The terms “controller,” “functionality,” “service,”and “logic” as used herein generally represent software, firmware,hardware, or a combination of software, firmware, or hardware inconjunction with controlling the mobile device 202. As shown in FIG. 1C,the controller 210 can include a processor 212, a memory 214, and acommunications interface 216.

The processor 212 provides processing functionality for at least thecontroller 210 and can include any number of processors,micro-controllers, circuitry, field programmable gate array (FPGA) orother processing systems, and resident or external memory for storingdata, executable code, and other information accessed or generated bythe controller 210. The processor 212 can execute one or more softwareprograms (e.g., mobile application 218) embodied in a non-transitorycomputer readable medium (e.g., memory 214) that implement techniquesdescribed herein. The processor 212 is not limited by the materials fromwhich it is formed or the processing mechanisms employed therein and, assuch, can be implemented via semiconductor(s) and/or transistors (e.g.,using electronic integrated circuit (IC) components), and so forth.

The memory 214 can be a tangible, computer-readable storage medium thatprovides storage functionality to store various data and or program codeassociated with operation of the controller 210, such as softwareprograms (e.g., mobile application 218 or “App”) and/or code segments,or other data to instruct the processor 212, and possibly othercomponents of the mobile device 202/controller 210, to perform thefunctionality described herein. The memory 214 can store data, such as aprogram of instructions for operating the mobile device 202 (includingits components), and so forth. It should be noted that while a singlememory 214 is described, a wide variety of types and combinations ofmemory (e.g., tangible, non-transitory memory) can be employed. Thememory 214 can be integral with the processor 212, can comprisestand-alone memory, or can be a combination of both. Some examples ofthe memory 214 can include removable and non-removable memorycomponents, such as random-access memory (RAM), read-only memory (ROM),flash memory (e.g., a secure digital (SD) memory card, a mini-SD memorycard, and/or a micro-SD memory card), magnetic memory, optical memory,universal serial bus (USB) memory devices, hard disk memory, externalmemory, and so forth. In embodiments, the mobile device 202 and/or thememory 214 can include removable integrated circuit card (ICC) memory,such as memory provided by a subscriber identity module (SIM) card, auniversal subscriber identity module (USIM) card, a universal integratedcircuit card (UICC), and so on.

The communications interface 216 can be operatively configured tocommunicate with components of the mobile device 202. For example, thecommunications interface 216 can be configured to transmit data forstorage in the mobile device 202, retrieve data from storage in themobile device 202, and so forth. The communications interface 216 canalso be communicatively coupled with the processor 212 to facilitatedata transfer between components of the mobile device 202 and theprocessor 212 (e.g., for communicating inputs to the processor 212received from a device communicatively coupled with the controller 210,including, but not limited to, data received from the locationdetermining component 224, any input device 226, and/or any othercomponent of the mobile device 202). It should be noted that while thecommunications interface 216 is described as a component of controller210, one or more components of the communications interface 216 can beimplemented as components of the mobile device 202 or componentscommunicatively coupled to the mobile device 202 via a wired and/orwireless connection. For example, the mobile device 202 and/or thecontroller 210 includes the short-range communications transceiver 220(or in some embodiments, a transmitter and a receiver) for sending andreceiving communications to and from the device 100 and/or vehicle audiosystem 204. The mobile device 202 and/or the controller 210 alsoincludes the cellular transceiver 222 for making phone calls and forsending and receiving communications to and from the voice service andnavigation/mapping servers 206 and 208.

As shown in FIG. 2C, the vehicle audio system 204 may also include acontroller 254 having a processor 256, memory 258, and communicationsinterface 260. The controller 254 can be configured to manageconnectivity with the device 100 and/or mobile device 202 for audiostreaming and/or hands-free calling as described herein. For example,the controller 254 can be configured to receive audio outputs from thedevice 100 and/or mobile device 202 via a short-range communicationstransceiver 262 (e.g., Bluetooth, NFC, and/or WiFi transceiver) of thevehicle audio system 204 and can be further configured to render theaudio outputs via a speaker system 266. In some embodiments, the vehicleaudio system 204 is configured to receive the audio outputs from thedevice 100 and/or mobile device 202 via an audio input interface 264(e.g., auxiliary input, USB input, or other audio connector/receiver).For example, the vehicle audio system 204 can be configured to receiveaudio from the device 100 and/or audio system 204 for streaming throughthe speaker system 266 via the audio input interface 264 instead of viaa short-range communications transceiver 262. In another example, thevehicle audio system 204 has both wired and wireless capabilities, wherethe vehicle audio system 204 is configured for wired (e.g., via theaudio input interface 264) and wireless (e.g., via the short-rangecommunications transceiver 262) connectivity based on user preference.As previously noted herein, there are a variety of configurations thatcan be implemented based on user preference and/or capabilities of thevehicle audio system 204 and/or mobile device 202 used in conjunctionwith the device 100 to implement the system 200.

The device 100 and/or system 200 can be configured with variousarrangements of components, programing, and so forth. In someembodiments, the device 100 and/or system 200 is/are configured bydisabling Bluetooth service delivery platform (SDP) advertised servicesfrom a client. The system 200 can include Bluetooth connections betweenthe device 100, mobile application 218 (e.g., via mobile device 202),and the vehicle audio system 204. In embodiments, depending on theuser's configuration settings in the mobile application 218, the device100 might not inform the mobile application 218 of all of itscapabilities so that the mobile application 218 may share processingresponsibilities between the controller 124 of the device 100 and thecontroller 210 of the mobile device 202 and/or the controller 254 of thevehicle audio system 254. The mobile application 218 may be configuredto cause the mobile device 202 to output a signal instructing the device100 to not advertise these services.

In some embodiments, the device 100 is configured to search for a pointof interest (POI) an intersection. For example, the device 100 may beconfigured to search for a POI near an intersection identified by theuser (e.g., “Find a Starbucks near 151st Street and Blackbob road”). Thedevice 100 can be configured to receive voice inputs (e.g., spokeninstructions and queries), sometimes without any other input methods, bycontinuously listening for a wake word (e.g., “Alexa,” “Siri,”“Cortana,” “Ok Google,” or the like). For example, the device 100 can beconfigured to provide location-based services and features (e.g.,services relying on a position determined by the location determiningcomponent 134 (e.g., GPS)) that are controlled by voice. The device 100may include buttons 108 (e.g., action and/or mic mute buttons, and soforth) or other input devices, but these other input devices may notneed to interact with the location determining functionality. Allcontrol and operation of location-based services and features may beenabled by voice inputs from the user. In embodiments where the device100 is limited to receiving voice inputs, the controller 124 ornavigation/mapping service 208 may be configured to implement atechnique to make search results usable by refining the query andselection process. For example, the user may ask the device 100 forassistance with navigating to a destination and the controller 124 ornavigation/mapping service 208 can be configured to provide the userwith search results one at a time—allowing the user to accept theresult, proceed to the next result, move back to the previous result, orstop the search without picking a result.

The device 100 may feature full compatibility with an intelligent voiceservice (e.g., via communication with server 206). For example, thedevice 100 may be compatible with the AMAZON ALEXA platform or any othervoice service that facilitates voice recognition accuracy to provideuseful digital assistant and multimedia features. Integrating voiceservices enables the controller 124 of the device 100 (e.g., via themobile device's 202 connectivity) to access to a full suite of features,such as a virtual assistant, music, podcasts, smart home integration,and custom skills. The device 100 builds on existing voice serviceplatforms by adding search, navigation, dash cam, and driver awarenessfeatures.

The device 100 can be configured to receive spoken instructions orqueries from a user to search for locations by using automatic speechrecognition (ASR) via the voice services server 206. In embodiments, thedevice 100 is configured to receive a spoken instruction or query from auser via the microphone 110 and then send data associated with thespoken instruction or query to the mobile application 218 running on themobile device 202 via the short-range communications transceiver 132.The mobile application 218 may then causes the mobile device 202 totransmit an audio output based on the spoken instruction or query viathe short-range communications transceiver 220 to the vehicle audiosystem 204 or to the device 100 (e.g., for output via an audio outputdevice (e.g., speaker 120 or audio output interface 144)). In someembodiments, the mobile application 218 is configured to cause themobile device 202 to transmit the data associated with the spokeninstruction or query to server 206 via the cellular transceiver 222 forvoice service processing and is further configured to receive navigationcommand data from server 206 based on the spoken instruction or query.The mobile application 218 can be further configured to transmit thenavigation command data and data associated with the current position ofthe device 100 (e.g., based on signals received via location determiningcomponent 134 or location determining component 224) to server 208 viathe cellular transceiver for navigation service processing. The mobileapplication 218 may be configured to receive information for the audiooutput (which can be provided via the device 100, vehicle audio system204, and/or mobile device 202) and/or a display output (e.g.,information which may be provided via the display 104 of the device 100)from server 208 in response to the navigation command data and the dataassociated with the current position of the device 100. In otherembodiments, the mobile application 218 can send both the dataassociated with the spoken instruction or query and the data associatedwith the current position of the device 100 to server 206 which mayperform a search based on the spoken instruction or query and thecurrent position of the device 100 and may then provide navigationcommand data in a format acceptable to server 208 directly or indirectly(e.g., via the mobile application 218) to server 208 which then returnsnavigation data for the device 100 (e.g., to generate one or more audiooutputs and/or display (e.g., symbolic and/or text) outputs). Byspeaking, the user may be able to quickly perform complex searches thatwould otherwise require several steps on a navigation device using atouch interface. For example, a user may ask, “Where can I get barbecuein Kansas City?” instead of having to select the appropriate menuoptions to select the area (city) of interest and category for a POIsearch.

FIG. 3 shows an example dialogue 300 between the user and the device100. For example, the user may ask “Where can I get barbecue in KansasCity?” and the device 100 may respond to the user query with a firstsuggestion 302 based on the current location of the device 100. Asshown, the user may provide a second query 304 for alternativesuggestions, and the device 100 can provide additional suggestions 306(e.g., places that are the next closest options or prioritized based onpast behavior, favorites, etc.). The device 100 can begin routenavigation when the user provides an instruction 308 based on a selecteddestination or acceptance of a suggestion provided by the device 100. Insome embodiments, server 206 processes the voice inputs (e.g., theuser's instructions/queries) and server 208 provides route guidanceinformation and other location services. There can be overlap infunctionalities. For example, server 206 may be configured to processvoice inputs and also perform search functions. Furthermore, some voiceinputs can be directly transferred to server 208 for navigation/mappingservices (e.g., when the user speaks a query that is known to the mobileapplication 218, e.g., “Where am I?”).

In implementations, the device 100 allows a user to find a POI based onname or category. For example, the user can say “Navigate to Starbucks”,“Navigate to Bobo Chinese”, or “Find the nearest gas station”. Inimplementations, the device 100 allows a user to find a POI based onrating. For example, the user may able to say “Find the best barbecuerestaurant nearby” and server 206 may be configured to identify andreturn highly-rated results stored in a memory (e.g., ratings fromFoursquare or TripAdvisor). The server 206 can be further configured toselect and implement techniques to enable the user to complete a name,category, or rating POI searches, by specifying one or more of thefollowing search locations: POIs Near a City—the user may be able to say“Navigate to Starbucks near Seattle” or “Find barbecue near KansasCity”; POIs Near an intersection—the user may be able to say “Find theChipotle near 95th and Metcalf” or “Find a pharmacy near 151st Streetand Antioch”; POIs Along Route—the user may be able to say “Find a gasstation ahead” or “Navigate to the next Starbucks up ahead”;Addresses—the user may be able to say “Navigate to 1200 East 151st St”or “1313 Harbor Blvd, Anaheim, Calif.”; City Search—the user may be ableto say “Navigate to Los Angeles”; and Saved Places—the user may be ableto say “Navigate home” and may also be able to store in a memory avariety of other pre-defined locations such as “work” and “school.”

The device 100 can improve real directions and direct access features toprovide spoken turn-by-turn instructions that do not require a displaydevice or any interaction with such a touchscreen. For example, FIG. 4shows an example device output 400 that includes a display (e.g.,symbolic and/or text) output 404 in addition to an audio output 402. Oneor more of the controllers (e.g., controller 124 and/or controller 210)may determine a display output 404 that includes a distance and an iconfor the next maneuver to reach a destination. The device controller 124can be configured to output audible voice instructions 402 and presentthe determined distance and icon 404 on the display 104 to assist theuser with traveling to the destination.

In embodiments, the display 104 may be small. The controller 124 can beconfigured to provide icons and text associated with a next maneuverinstead of providing full guidance information due to the limitedinformation that may be presented on a small display. Example displayoutputs are illustrated in FIGS. 5A through 5H. For example, FIG. 5Aillustrates pre-turn preparation; FIG. 5B illustrates a simple turnarrow; FIGS. 5C and 5D illustrate lane assist guidance; FIG. 5Eillustrates end of route guidance; FIGS. 5F and 5G illustrate laneassist guidance with identification of a street name; and FIG. 5Hillustrates end of route guidance with a house number.

At the start of a route, the device 100 can be configured to presentinformation to help clarify a major road that may be taken, to help theuser understand which way the route is going. For example, FIG. 6 showsan example device output 600 that includes a display (e.g., symbolicand/or text) output 604 in addition to an audio output 602. One or moreof the controllers (e.g., controller 124 and/or controller 210) maydetermine a display output 604 that includes a distance and an icon forthe next maneuver to reach a destination. The device controller 124 canbe configured to output audible voice instructions 602 includinginformation regarding the route (e.g., major roads), destination,estimated time of arrival (ETA), and/or other route information for theuser when the route guidance begins or is about to begin.

When the user starts a route to a destination in a parking lot or otheroff-road or unmarked location, the controller 124 may be configured tooutput audible instructions providing detailed instructionssupplementing the information presented on the display 104 to assist theuser with starting the determined route. For example, FIG. 7 shows anexample device output 700 that includes a display (e.g., symbolic and/ortext) output 704 in addition to an audio output 702. One or more of thecontrollers (e.g., controller 124 and/or controller 210) may determine adisplay output 704 that includes a distance to a road at which the routeguidance begins. The device controller 124 can also be configured tooutput audible voice instructions 702 that instruct the user to head tothe road at which the route guidance begins. The display output 704 mayalso include an arrow pointing towards the road to assist the user. Insome embodiments, the controller 124 is configured to present anillumination pattern 706 via the indicator lights 106 to point towardsthe road (e.g., in addition to or instead of a pointer on the display104).

In densely-populated cities (having many roads in a geographic area), itcan be challenging to guide a user through a route on roads that arevery close to other roads. The controller 124 can be configured tocontrol the display 104 and the indicator lights 106 to illustrate whento take a turn or remain on the current road. For example, FIGS. 8A and8B show examples of a device output 800 that includes a display (e.g.,symbolic and/or text) output 804 in addition to an audio output 802. Oneor more of the controllers (e.g., controller 124 and/or controller 210)may determine a display output 804 that includes a distance to a turn.The device controller 124 can also be configured to output audible voiceinstructions 802 that instruct the user to turn with a distance to turnand detailed information (e.g., “the second turn on the right”). As theuser nears the turn, the controller 124 can be configured to generate anoutput 806 that provides a detailed and/or zoomed in view of which turnthe user should be making. Additionally, the controller 124 can beconfigured to present a dynamic illumination pattern via the indicatorlights (e.g., by filling in the ring with light or a particular color)to indicate proximity of the vehicle to the turn. This can help ensurethat the user does not make a turn too early or too late (e.g., onto awrong street).

It is common for a user to go off-route when driving to a destination,for example, when traveling around a parking lot near the end of aroute. In these situations, the controller 124 can be configured todisplay a distance to the destination and provide an illuminationpattern via the indicator lights 106 to guide the user to thedestination. For example, FIG. 9A shows an example of a device outputtowards the end of the route. In embodiments, the controller 124 isconfigured to initiate an arrival mode when the current position is lessthan a threshold distance from the destination or the current positioncorresponds to an unmarked location. As shown in FIG. 9A, when in thearrival mode, the controller 124 can control the indicator lights 106 topresent an illumination pattern 900 that acts as a bearing pointer tohelp point the user to the destination. The controller 124 can also beconfigured to cause the display 104 to display a distance to thedestination and/or an arrow that points the user to the destination. Insome embodiments, the controller 124 is configured to cause the display104 to present an arrow that points the user to the destination insteadof presenting an illumination pattern via the indicator lights 106.

In some embodiments, the device 100 includes a camera 122 that may facein front of the vehicle when the device 100 is oriented accordingly. Thecontroller 124 can be configured to utilize data (e.g., images and/orvideo footage) from the camera 122 to provide a number of features. Forexample, the device 100 can be configured to record video footage orimages of the area in front of the vehicle continuously, periodically,or on demand In embodiments, the controller 124 is configured to savethe images or video footage in a memory (e.g., memory 128 of the device100, memory 214 of the mobile device 202, or in a remote server (e.g.,server 206, 208, or a data storage server). In some embodiments, thecontroller 124 is configured to save the images or video footage to thememory when an incident is detected (e.g., a collision, abrupt stop,etc.) or after receiving a voice input from the user instructing thedevice 100 to save the images or video footage. In some embodiments, thedevice 100 is further configured to save a “time-lapse” entry includingstored footage, location, and/or time information. The device 100 can beconfigured to store the time-lapse entries to a server to enable sharingof a user's journey with friends or on social media.

The device 100 may control the display 104, indicator lights 106, and/orspeaker 120 (or audio output interface 144) to notify the user when theuser is too close to a car in front of the user's vehicle, when the useris driving too close to another lane, when the user is not moving withtraffic, and so forth.

FIG. 9B illustrates an example embodiment where the controller 124 isconfigured to detect when there is less than a threshold proximity(e.g., less than a threshold distance) to an object in a field of viewof the camera 122 based on the image(s) and/or video recorded by thecamera 122 while the device 100 is in motion, and the controller 124further configured to cause the plurality of indicator lights 106 todisplay a forward collision warning illumination pattern 902 (e.g.,lighting up or flashing the light ring or a portion (e.g., frontportion) of the light ring) in response to detecting less than athreshold proximity to an object in the field of view of the camera 122.The controller 124 can also be configured to provide a forward collisionwarning display output 904 and/or audio output 906 to alert the user.Another example is shown in FIG. 12A, where the front portion 1200 ofthe light ring defined by the indicator lights 106 is illuminated and aforward collision warning display icon 1202 is presented via the display104.

FIG. 9C illustrates another example embodiment where the controller 124is configured to detect when there at least a threshold deviation from areference alignment between lanes in a field of view of the camera 122based on the image(s) and/or video recorded by the camera 122 while thedevice 100 is in motion, and the controller 124 further configured tocause the plurality of indicator lights 106 to display a lane departurewarning illumination pattern 908 (e.g., lighting up or flashing theright or left side of the light ring) in response to detecting at leasta threshold deviation from a reference alignment between lanes in afield of view of the camera 122. The controller 124 can also beconfigured to provide a lane departure warning display output 910 and/oraudio output 912 to alert the user. Another example is shown in FIG.12B, where the left portion 1204 of the light ring defined by theindicator lights 106 is illuminated to indicate a left-side lanedeparture and a left-side lane departure warning display icon 1206 ispresented via the display 104.

As shown in FIG. 9D, the controller 124 can also be configured toprovide an audio output 914 and/or display output to alert the user thattraffic has begun moving. For example, the controller 124 can beconfigured to determine that objects (e.g., other vehicles) in the fieldof view of the camera 122 have are no longer in view or are at least athreshold distance from the camera 122, and the controller 124 can beconfigured to provide the audio output 914 and/or display output toalert the user that traffic has begun moving in response to determiningthat the objects are no longer in view or are at least a thresholddistance from the camera 122. Oftentimes, a user may be distracted whilesitting at a traffic light and not realize that a car ahead of theuser's vehicle has started moving. Accordingly, providing an audio alert914 for the user can help refocus the user's attention.

In embodiments, the device 100 can be configured to receive speed limitinformation associated with a current location of the device 100 (e.g.,a speed limit on a current road traversed by the user's vehicle) anddetermine when the speed limit reduces on the current road ahead. Forexample, the device 100 can be configured to receive this informationfrom server 208. The controller 100 may be configured to present a speedreduction warning (e.g., flashing yellow light, or the like) via theindicator lights 106 to alert the user that the current road has areduced speed limit ahead. As shown in FIG. 9E, the controller 124 canalso be configured to provide a display output 916 and/or audio output918 to alert the user of an upcoming speed reduction. In someembodiments, the memory 128 of the device 100 itself may include maps ormap data with speed limit information to provide this warning duringnavigation or at any other time. When the device 100 is determined to bein a vehicle traveling at a speed in excess of (e.g., over) the speedlimit, the controller 124 may control the display 104 and/or theindicator lights 106 to alert the driver, such as with a subtleindicator (e.g., flashing display output and/or change in color of theindicator lights 106). In embodiments, the controller 124 can also beconfigured to provide an audible warning via the speaker 120 and/oraudio output interface 144. The speed of the vehicle may be determinedbased on change in position (e.g., based on signals received via thelocation determining component 134), based on camera 122 footage, basedon information received from the vehicle, or based on a speed detectedby a speed sensor in the device 100.

The device 100 may support a variety of features commonly used ineveryday drives. For example, the device 100 may be configured toprovide access to music, podcast, audio book services, news, and othermedia stored remoted on remote servers (e.g., server 206), such asAMAZON servers, GOOGLE servers, YAHOO servers, and the like. The device100 may also access media services, such as, but not limited to,PANDORA, SPOTIFY, SOUNDCLOUD, ITUNES, and GOOGLE PLAY.

The device 100 may support various “smart notifications” that may appearwith a visual indicator to minimize disruption to a user with audio or avoice prompt. For example, the user may ask the device 100 to read anincoming message and then vocally instruct the device to clear or deletethe message, reply, read the next message, access previously storedmessages, and so forth. In embodiments, different notifications may beturned on/off via the mobile application 218 on the mobile device 202based on user preference and/or to prevent distractions while the useris driving (e.g., during route guidance).

The device 100 may connect to a remote server (e.g., server 206, 208, oranother server) that provides weather information and service. Inembodiments, the controller 124 and/or controller 210 may be configuredto transmit the current location of the device 100 (e.g., the currentgeographic position determined by the location determining component134) and/or route information (e.g., navigation data) to the remoteserver to receive weather information from the remote server based uponthe current location of the device 100 and/or route information (e.g.,weather conditions at the current location or between the currentlocation and the destination).

The device 100 can also be configured to receive traffic updates. Forexample, the device 100 may be configured to periodically receive orrequest updated traffic information from a remote server (e.g., server206, 208, or another server). In embodiments, controller 124 and/orcontroller 210 may be configured to transmit the current location of thedevice 100 and/or route information (e.g., navigation data) to theremote server to receive updated traffic information from the remoteserver based upon the current location of the device 100 and/or routeinformation (e.g., traffic conditions at the current location or betweenthe current location and the destination). In some embodiments,controller 124 and/or controller 210 may be configured to receivetraffic information from the remote server automatically if traffic isdetermined to exist along a current route to a destination. Controller124 and/or controller 210 may be configured to receive navigation datafrom the remote server (e.g., server 208) based on the updated trafficinformation to calculate a route to the destination. For example, theroute may be at least partially based on (e.g., optimized for) thecurrent traffic conditions. In some embodiments, the user may able toask, “How's the traffic ahead”. The device 100 may connect to a serverassociated with a traffic provider other than the voice services server206, such as the navigation/mapping services server 208, to determinecurrent traffic conditions and control the display 104, indicator lights106, and speaker 120 to provide the traffic information.

In embodiments, the device 100 is also configured to connect with aremote server (e.g., server 206 and/or 208) to determine answers toquestions such as, “Where am I?”, to obtain the name of the road beingdriven on, and “What's the speed limit?”, to obtain the current speedlimit of that road (if stored map data contains that information).

As previously noted herein, the device 100 can include one or more ofthe following forms of output: an indicator light 106 (e.g., an LEDring), a display 104 (e.g., LED/OLED display), and a speaker 120 oraudio output interface 144. The controller 124 is configured to controlany of the output devices 140 (e.g., indicator lights 106, display 104,speaker 120 and/or audio output interface 144) to provide the user withinformation, alerts, responses to queries, message notifications,calendar notifications, and so forth.

The speaker 120 and/or audio output interface 144 outputs audibleinformation and provides a “voice” for the device 100. The controller124 is configured to employ the speaker 120 and/or audio outputinterface 144 to respond to voice queries, providing navigationinstructions, driver assistance alerts, and in-car entertainment withmusic, audio books, or podcasts. The spoken instructions or queries canbe sent to server 206 for voice services processing, where the server206 is configured to return responsive communications to the device 100and can also be configured to translate the user's natural languageinstructions/queries into command data formatted for anavigation/mapping server 208 or any other server that server 206 isworking with to answer the user's query or fulfill the user'sinstruction. In some implementations, spoken information may be given inthe “ALEXA” voice by querying AMAZON's cloud-based, text-to-speech (TTS)engine, and the command data can include instructions formatted for thenavigation/mapping server 208 (e.g., GARMIN's cloud-basednavigation/mapping services)). The priority of information output by thespeaker 120 and/or audio output interface 144 may be as shown in FIG.10, for example, the priority chain 1000 can be as follows: (1) ASRresponses 1002; (2) navigation instructions 1004; (3) alerts andwarnings (e.g., ADAS or other driver alerts) 1006, (4) Music or Podcasts1008, with 1 being the highest priority and 4 being the lowest.

The controller 124 can be configured to present various illuminationpatterns via the indicator lights 106 (e.g., an LED light ring) toenhance and visualize information provided in the ASR conversation.Different illumination patterns may be implemented to provideinformation, such as different states for use when the device 100 isspeaking, listening, and thinking. Controller 124 and controller 210 maywork closely with the voice processing server 206 to provide aconsistent experience between the device 100 and a compatible product(e.g., a smart home assistant, such as AMAZON ECHO, or the like). Inembodiments, the indicator lights 106 define an LED light ring thatincludes a plurality of (e.g., 8, 10, 12, 14, 16, etc.) individuallyselectable/controllable light segments.

FIG. 11 shows an example process 1100 for interacting with the device100 that employs the indicator lights 106 to indicate device states tothe user. The device 100 listens for the user to speak a wakeup word(block 1102). Until the user does so, the indicator lights 106 may beturned off completely or set to a standby/inactive illumination pattern(block 1104). When the user speaks a wake word (e.g., “Device”, “Alexa,”“Siri,” “Cortana,” “Ok Google,” or the like), the device can present anillumination pattern to indicate that the device is listening (block1106). If the spoken instruction/query is too quiet, the device 100 cancontinue to listen (block 1106). If the spoken instruction/query is atleast a threshold volume, the device communicates the audio to the voiceservices server 206 (block 1108). For example, the device 100 can beconfigured to transmit the spoken instruction/query to the mobileapplication 218 running on the mobile device 202, where the mobileapplication 218 causes the mobile device 202 to communicate the spokeninstruction/query to the server 206. The device 100 may provide anillumination pattern (e.g., spinning/flashing the light ring) toindicate processing of the spoken instruction/query by the server 206(block 1110). When the server 206 sends a response, the device 100 cancommunicate the response to the user while displaying anotherillumination pattern (e.g., a solid blue light ring) via the indicatorlights 106 (block 1112). The device 100 can then listen for any followup instructions/queries from the user (block 1114). When the device isdone communicating with the user (e.g., after a threshold time haspassed with no spoken instructions/queries from the user), the devicereturns to a standby/inactive state, where the controller 124 turns offthe indicator lights 106 or presents a standby/inactive illuminationpattern (block 1116).

In addition to indicate device 100 states, the controller 124 can beconfigured to present illumination patterns via the indicator lights toprovide ADAS alerts/warnings (e.g., a forward collision warning as shownin FIG. 12A, a lane departure warning as shown in FIG. 12B, speed limitwarnings, traffic warnings, and the like). In embodiments, thecontroller 124 is configured to present an illumination pattern via theindicator lights 106 to indicate a current bearing, recommendeddirection along a route to a destination, or a compass (pointing North).

The controller 124 is also be configured to provide display outputs(e.g., symbols and/or text) via the display 104 to compliment the audiooutputs provided via the speaker 120 and/or audio output interface 144and/or the illumination patterns presented via the indicator lights 106.For example, the display 104 can be used to provide route guidanceinformation, ADAS alerts, notifications, media information, and soforth. The priority of information presented via the display 104 may beas shown in FIG. 13, for example, the priority chain 1300 can be asfollows: (1) ASR responses and/or requested information 1302; (2) smartnotifications (e.g., incoming calls, message notifications, etc.) 1304;(3) navigation/guidance data for current or upcoming maneuvers (e.g.,navigation near a turn, and so forth) 1306, (4) alerts and warnings(e.g., ADAS or other driver alerts or warnings 1308, (5)navigation/guidance in advance of a maneuver (e.g., information about aturn that is a long distance (e.g., 1 mile or more) from the currentposition) 1310, with 1 being the highest priority and 5 being thelowest.

FIGS. 14 through 36B show various example processes and use scenariosthat illustrate device 100 configurations. Unless otherwise stated, thedevice can be configured to execute the operational blocks illustratedin FIGS. 14 through 36B in any arbitrary order. Furthermore, operationalblocks of different processes can be combined and are provided herein todemonstrate various configurations of the device 100 but are notintended as limitations. Embodiments/implementations described hereincan be fully or partially combined to achieveembodiments/implementations other than those that are explicitlyprovided herein as a single embodiment/implementation and/or illustratedin a single drawing or series of drawings of FIGS. 14 through 30B. Theoperational blocks can be executed by any combination of hardware,software, or firmware, for example, operations completed by the device100 actions can be executed at least in part by the controller 124 andin some embodiments, by the mobile device controller 210 and/or a server(e.g., server 206 and/or server 208).

FIG. 14 shows an example process 1400 for interacting with the device100. The device 100 listens for the user to speak a wakeup word or pressthe action button 108 (block 1402). Until the user does so, theindicator lights 106 may be turned off completely or set to astandby/inactive illumination pattern (block 1404). When the user speaksa wake word (e.g., “Device”, “Alexa,” “Siri,” “Cortana,” “Ok Google,” orthe like) or presses the action button 108, the device can present anillumination pattern to indicate that the device is listening and mayalso output a listening tone via the speaker 120 (block 1406). If thespoken instruction/query is too quiet, the device 100 can continue tolisten (block 1406). If the spoken instruction/query is at least athreshold volume, the device communicates the audio to the voiceservices server 206 (block 1408). For example, the device 100 can beconfigured to transmit the spoken instruction/query to the mobileapplication 218 running on the mobile device 202, where the mobileapplication 218 causes the mobile device 202 to communicate the spokeninstruction/query to the server 206. The device 100 may provide anillumination pattern (e.g., spinning/flashing the light ring) toindicate processing of the spoken instruction/query by the server 206(block 1410). When the server 206 sends a response, the device 100 cancommunicate the response to the user while displaying anotherillumination pattern (e.g., a solid blue light ring) via the indicatorlights 106 (block 1412). The device 100 can then listen for any followup instructions/queries from the user (block 1414). If the user pressesthe action button or speaks the wakeup word again, the device starts anew ASR session (block 1418) and returns to block 1406 to listen for anew instruction/query from the user. When the device is donecommunicating with the user (e.g., after a threshold time has passedwith no spoken instructions/queries from the user), the device returnsto a standby/inactive state, where the controller 124 turns off theindicator lights 106 or presents a standby/inactive illumination pattern(block 1416). The microphone 110 may be muted (e.g., the device 100ignores all spoken instructions/queries) until the wakeup word is spokenor the action button is pressed again (block 1420).

FIGS. 15A through 15D show an example process 1500 for interacting withthe device 100 to search for a destination and/or initiate routeguidance. The user may employ the device 100 to find a location (block1502). The device 100 is plugged in, powered on, and waiting for theuser to speak the wakeup phrase or press the action button 108 (block1504). When the user speaks a wake word (e.g., “Device”, “Alexa,”“Siri,” “Cortana,” “Ok Google,” or the like) or presses the actionbutton 108, the device can present an illumination pattern to indicatethat the device is listening and may also output a listening tone viathe speaker 120 (block 1506). The device 100 can receive a spokeninstruction or query (e.g., search query) from the user and send thespoken instruction or query to the server 206 for voice serviceprocessing (block 1508). The controller 124 or mobile application 218determines whether route guidance is already active (block 1510). Whenroute guidance is not active, the server 206 can locate results based onthe current location (e.g., sorted based on nearness to the currentlocation) of the device 100, or based on a specified search location(e.g., in a particular city, near an intersection, zip code, etc.)(block 1512). When route guidance is active, the server 206 can locateresults based on the route being followed by the device 100 (e.g., theserver 206 can search for places located along the active route) (block1514).

When the server 206 completes the search (block 1516/1518), the device100 may determine whether or not the bearing of the device 100 is known(e.g., based on the movement of the device 100) (block 1520). If thebearing is not known, the device 100 can provide a description of thelocation via an audio and/or display output and can also provide thedirection to reach the location (block 1522). If the bearing is known,the device 100 can provide a description of the location via an audioand/or display output and can also provide the direction to reach thelocation relative to the current direction of travel (e.g., bydisplaying an arrow or illumination pattern that points towards thelocation relative to the bearing of the device 100) (block 1524). If theuser provides a voice input to stop or discontinue the search, thedevice 100 can end the ASR session and resume operation (e.g., byreturning to a standby/inactive state or resuming active route guidance)(block 1530). If the user provides a voice input that accepts thesuggested location and/or instructs the device 100 to navigate to thelocation, the device 100 may determine if route guidance is alreadyactive (block 1526). If route guidance is not already active, the device100 may send navigation command data (e.g., data received from server206 based on the user's voice inputs) to server 208 for navigationservice processing and can receive navigation data (e.g., route guidanceinformation) from server 208 in response (block 1540). If route guidanceis already active, the device 100 may ask the user whether to start anew route or add the search location to the current route (block 1532).The user may instruct the device 100 to start a new route, where thedevice 100 then stops the current route and starts a new route based onthe search location (block 1534). The user may instruct the device 100to navigation to the search location next, where the device 100 thenadds the search location as the next stop and updates the current routeaccordingly (block 1536). The user may instruct the device 100 tonavigation to the search location last, where the device 100 then addsthe search location as the last stop and updates the current routeaccordingly (block 1538). After receiving user instructions and updatingthe current route or starting a new route, the device 100 may sendnavigation command data (e.g., data received from server 206 based onthe user's voice inputs) to server 208 for navigation service processingand can receive navigation data (e.g., route guidance information) fromserver 208 in response (block 1540).

The device 100 may provide an audio and/or display output to let theuser know that the device 100 is getting directions from the server 208(block 1540), and in some implementations, the device 100 provides anillumination pattern (e.g., spinning blue ring) via the indicator lights106 while the route calculation is in progress (block 1542). The device100 may assess whether or not there is satellite reception (e.g., GPSreception) (block 1544). If there is no reception, the device 100 maynotify the user that it is waiting for improved reception before routeguidance can begin (block 1552). In some instance, the device 100position may change (e.g., because the user continues driving) while thedevice 100 is waiting for reception. The device 100 may need torecalculate the route if the position changes too much (block 1554).When the device 100 has sufficient satellite reception, the device 100determines whether or not the route can be calculated to the destinationbased on the navigation data (e.g., route guidance instructions)received from the server 208 (block 1546). If there is an error (e.g.,if the device 100 cannot calculate the route), the device 100 may beconfigured to provide a display and/or audio output notifying the userthat the route cannot be calculated (block 1556). When the route iscalculated, the device 100 is configured to start navigationinstructions (block 1550). The device 100 can also provide trafficupdates and/or ADAS alerts in addition to providing the navigationinstructions to the user (block 1548).

FIGS. 16A and 16B show an example process 1600 for interacting with thedevice 100 to search for a destination and/or initiate route guidancevia the mobile device 202 (e.g., via mobile application 218). The cansend a location (e.g., manually entered destination or search location)from the mobile device 202 (e.g., via mobile application 218 and/oranother linked mobile application) to the device 100 while the device ison (block 1602) or while the device 100 is off for transfer to thedevice 1604 when the device 100 is turned on and connected to the mobiledevice 202 (block 1604). The device 100 may provide an audio and/ordisplay output to let the user know that the device 100 is gettingdirections from the server 208 (block 1606), and in someimplementations, the device 100 provides an illumination pattern (e.g.,spinning blue ring) via the indicator lights 106 while the routecalculation is in progress (block 1608). The device 100 may assesswhether or not there is satellite reception (e.g., GPS reception) (block1610). If there is no reception, the device 100 may notify the user thatit is waiting for improved reception before route guidance can begin(block 1614). In some instance, the device 100 position may change(e.g., because the user continues driving) while the device 100 iswaiting for reception. The device 100 may need to recalculate the routeif the position changes too much (block 1616). When the device 100 hassufficient satellite reception, the device 100 determines whether or notthe route can be calculated to the destination based on the navigationdata (e.g., route guidance instructions) received from the server 208(block 1612). If there is an error (e.g., if the device 100 cannotcalculate the route), the device 100 may be configured to provide adisplay and/or audio output notifying the user that the route cannot becalculated (block 1618). When the route is calculated, the device 100 isconfigured to start navigation instructions (block 1622). The device 100can also provide traffic updates and/or ADAS alerts in addition toproviding the navigation instructions to the user (block 1620).

FIGS. 17A and 17B show an example of voice inputs that can be handled byserver 206 for location searching. For example, the user can requestthat the voice service (e.g., service 206) ask the navigation service(e.g., server 208) to find an address, POI, name of a place, etc. In anexample shown in FIG. 17A, the user asks the device 100 to find a coffeeshop. The device 100 can transmit the spoken instruction/query receivedfrom the user to server 206 for voice service processing, and server 206can communicate with server 208 for navigation/mapping serviceprocessing to locate nearby coffee shops by communicating navigationcommand data (e.g., instructions based on ASR processing of the user'svoice input) directly or indirectly (e.g., by sending the navigationcommand data back through the mobile application 218 to server 208). Thedevice 100 can then provide search results one at a time for the uservia audio and/or display outputs and can listen for user responses(e.g., accepting or rejection the search results, asking foralternatives, etc.). As shown in FIG. 17B, the device 100 can handlemore specific requests. For example, the user can ask the device tolocate a POI in a city or near a location/landmark. For instance, in theexample shown in FIG. 17B, the user asks the device 100 to find a coffeeshop near a particular city. FIGS. 17A and 17B are provided toillustrate examples of the user-device interaction. In implementations,the device 100 can accept additional/alternative voice inputs from theuser. For example, the user can search for locations by name, address,city, state, neighborhood, landmark, distance, category, and so forth.The device 100 acts as a gateway between the user and servers 206 and208 for ASR services and navigation/mapping services.

FIGS. 18A and 18B show an example process 1800 for interacting with thedevice 100 to search for a location and navigate through a plurality ofsearch results. After the wakeup word is spoken or the action button ispressed, the device 100 can listen for a spoken instruction or queryfrom the user (block 1802). For example, the user can ask the device 100to search for a nearby coffee shop. The device 100 can then send thespoken instruction or query to server 206 for voice processing, and insome implementations, server 206 is configured to return results and/orcommunicate with a navigation/mapping server 208 to complete the search.The device 100 may be configured to send its current location to server206 and/or server 208 so that the search results are at least partiallybased on the current location of the device 100 (e.g., the server206/208 returns search results for coffee shops based on nearness to thedevice 100 location). The device 100 may output results one by one, orseveral at a time (block 1804). The device 100 can be configured to moveto the next result if the user rejects the search result presented bythe device 100 as an audio and/or display output (block 1806). Thedevice 100 can continue to present results each time the user rejects aresult or asks for more results (blocks 1808, 1814, 1816, and 1818). Thedevice 100 may provide an audio and/or display output notifying the userthat there are no more results after the device 100 has provided all ofthe results received from the server 206 and/or after presenting morethan a threshold number of results (e.g., after presenting twenty (20)results) (block 1820). The device 100 can then end the ASR session(block 1822). If the user wants to hear or view previously presentedresults, the device 100 can present the results again when the user asksthe device 100 to produce a previous result. The user may be able tocycle through the results by saying “next” and “previous.” For example,after presenting results at block 1806, the user may view a previouslypresented result by requesting the previous result from the device 100(block 1812). If the user says previous too many times, the device 100may be configured to provide an audio and/or display output notifyingthe user that there are no more previously presented results (block1810).

FIG. 19 shows an example process 1900 for interacting with the device100 to search for a previously saved location. After the wakeup word isspoken or the action button is pressed, the device 100 can listen for aspoken instruction or query from the user (block 1902). For example, theuser can ask the device 100 to navigate to “home.” The device 100 maycheck with the mobile application 218, server 206, and/or server 208 todetermine if the location has been previously saved (block 1904). Forexample, the user may have previously saved a home address via thedevice 100 and/or mobile application 218. If the location has not beensaved, the user can optionally use the mobile application 218 to savethe location (block 1910). Examples of locations that can be savedinclude, but are not limited to, the user's “home,” “work,” “school,”addresses of contacts, favorites, and so forth. If the locationrequested by the user corresponds to a previously saved location, thedevice 100 may provide an audio and/or display output to notify the userthat the device 100 will begin route guidance to the requested location(block 1906). The device 100 can then begin route guidance (block 1908).

FIGS. 20A and 20B show an example process 2000 for presenting routeguidance information via the device 100 (e.g., via the display 104,speaker 120/audio output interface 144, and/or indicator lights 106).Route guidance on the device 100 can be active (block 2002). The device100 may present symbolic and/or text outputs (e.g., a maneuver icon anddistance to the maneuver (e.g., a turn)) via the display 104 (block2004). The device 100 can also provide one or more audio outputs tonotify the driver of the upcoming maneuver (block 2006). In someimplementations, the device 100 provides an audio output (e.g., pre-turnpreparation output) when it first displays the upcoming maneuver to theuser and again when the user is less than a threshold distance from themaneuver (block 2008). If the distance to the maneuver is short, thedevice 100 may skip the pre-turn preparation audio output. The device100 may present a dynamic illumination pattern via the indicator lights106 (e.g., gradually filling in portions of the light ring) when theuser is within the threshold distance from the maneuver (block 2010).For example, the device 100 can begin to illuminate a bottom portion ofthe light ring defined by indicator lights 106 when the user is withinthe threshold distance from the maneuver (block 2012). The device 100may also provide an audio output letting the user know the distance fromthe maneuver. The device 100 can continue to fill in portions of thelight ring as the user nears the maneuver (block 2014), and can provideadditional audio outputs when the user is within viewing range (e.g.,less than X feet) from the maneuver (block 2016). The device 100 maycontinue to fill in the light ring until it is completely filled in(e.g., when the user reaches the maneuver) (blocks 2018 and 2020). Thedevice 100 may then proceed to present a display, audio, and/orillumination pattern output for the next maneuver (block 2022).

FIGS. 21A and 21B show an example process 2100 for presenting routeguidance information via the device 100 (e.g., via the display 104,speaker 120/audio output interface 144, and/or indicator lights 106)when the device 100 is off-route (e.g., in a parking lot, off-roadlocation, or at an unmarked/unmapped position). The device 100 cancalculate and/or receive navigation data/instructions (e.g., from server208) for route guidance (block 2102). The device 100 may determine ifthe device 100 is less than a predefined maximum distance (e.g., lessthan X meters) from the route (block 2104). If the device 100 is lessthan the predefined maximum distance from the route, the device 100 canbegin/continue route navigation (block 2106). The device 100 can displayadditional maneuvers (e.g., next turn, etc.) when the device 100 beginsto travel along the route (block 2106). If the device 100 is more thanthe predefined maximum distance from the route, the device 100 cannavigate the user to the route. For example, the device 100 can providean audio and/or display output to navigate the user to the route (block2110). The device 100 may provide a bearing pointer via the display 104to assist the user (block 2112). For example, the device 100 can providean arrow pointing towards the route and may also display the distance tothe route (block 2114). Map view 2120 shows an example of the device 100location relative to the route. If the user drives to a new positionthat is farther from the route starting point indicated by the device100 (e.g., as shown in map view 2122), the device 100 may provide abearing pointer via the display 104 to navigate the user to a different(closer) starting point along the route based on the new position of thedevice 100 (block 2116). For example, the device 100 can provide anarrow pointing towards the new starting point along the route and mayalso display the distance to the new starting point (block 2118).

FIG. 22 shows an example process 2200 for presenting route guidanceinformation via the device 100 when the device 100 is in an environmentwith clustered (e.g., back-to-back) turns, for example, indensely-populated cities (having many roads in a geographic area). Theuser may employ the device 100 for route guidance in situations where anupcoming turn requires the user to turn on to one of many back-to-backroads (block 2202). The device 100 can provide an audio and/or displayoutput to guide the user to turn onto the correct road with detailedinstructions. For example, the device 100 can provide an audio output tonotify the user of the correct turn (e.g., block 2204—“the second turnon the right” or block 2206—“the first turn on the right”). As the usernears the turn, the device 100 can also provide a display output thatprovides a detailed and/or zoomed in view of which turn the user shouldbe making. For example, the device 100 can provided a highlighted viewof the second turn on the right (block 2208) or a highlighted view ofthe first turn on the right (block 2210). Additionally, the device 100may be configured to present a dynamic illumination pattern via theindicator lights 106 (e.g., by filling in the ring with light or aparticular color) to indicate proximity of the vehicle to the turn. Thiscan help ensure that the user does not make a turn too early or too late(e.g., onto a wrong street).

The user can query the device 100 for information regarding the currentroute and/or location of the device 100. For example, FIG. 23 shows anexample process 2300 for interacting with the device 100 to inquireabout the speed limit at a current location along a route. After thewakeup word is spoken or the action button is pressed, the device 100can listen for a spoken instruction or query from the user regarding thespeed limit at the current position of the device 100 (block 1802). Thecontroller 124 or mobile application 218 may first determine whether aroute is active (e.g., is the device 100 moving?) (block 2304). If thedevice 100 is on an active route, the controller 124 or mobileapplication 218 may query server 208 for the speed limit at the currentposition of the device 100 (block 2306). If the server 208 is unable toreturn a response or indicates that the speed limit at the currentposition is not known, the device 100 can provide an audio and/ordisplay output to inform the user that the speed limit at the currentposition of the device 100 cannot be determined (block 2308). If theserver 208 responds to the device 100 with a speed limit, the device 100can be configured to provide an audio and/or display output to informthe user of the speed limit at the current position of the device 100(block 2310). The device 100 may display the speed limit for a period oftime (e.g., X seconds) (block 2310) and can return to its previousdisplay or turn off the display after the period of time has passed(block 2314).

FIG. 24 shows an example process 2400 for presenting route guidanceinformation via the device 100 (e.g., via the display 104, speaker120/audio output interface 144, and/or indicator lights 106) when thedevice 100 is off-route (e.g., in a parking lot, off-road location, orat an unmarked/unmapped position) and close to (e.g., less than X milesfrom) a destination. The device 100 may present active route guidance asthe user approaches the destination (block 2402). The device 100 mayprovide an audio and/or display output to inform the user of a detailedposition of the destination (e.g., “destination is up ahead on theright”). As the user approaches the destination, the controller 124 ormobile application 218 can track the user's distance from thedestination to determine when the user is within threshold proximity tothe destination (e.g., less than M meters from the destination) (block2404). The controller 124 or mobile application 218 may then determinethat the user has effectively arrived at the destination and can endroute guidance (block 2406). If the user drives off the route whileapproaching the destination, the controller 124 or mobile application218 may recalculate the route (block 2408). After recalculating theroute, the controller 124 or mobile application 218 may determinewhether the user is close to (e.g., less than X miles from) thedestination and whether the user is at a marked/unmarked location (e.g.,whether or not the user is on an identifiable street or an unnamedlocation, such as a parking lot, off-road location, etc.) (block 2410).If the user is at a marked location (e.g., identifiable street) and/ormore than the threshold distance from the destination, the device 100can continue route guidance (block 2412). In some implementations, thedevice 100 can provide the user with an audio and/or display output thathelps guide the user to the route (e.g., as described with reference toFIGS. 21A and 21B). If the user is at an unmarked location, thecontroller 124 or mobile application 218 can initiate an arrival mode2414. In arrival mode, the device 100 may display a distance to thedestination and/or an arrow 2417 that points the user to the destination(block 2416). The device 100 may also provide an audio output and/orpresent an illumination pattern via the indicator lights 106 that actsas a bearing pointer (e.g., instead of or in addition to the arrow 2417)to help point the user to the destination.

FIG. 25A shows another example process 2500 for presenting routeguidance information via the device 100 (e.g., via the display 104,speaker 120/audio output interface 144, and/or indicator lights 106)when the device 100 is approaching a destination. The device 100 maypresent active route guidance as the user approaches the destination,where the final navigation instruction causes the device 100 to gooff-route (e.g., into a parking lot) and enter arrival mode (block2502). For example, device 100 may provide an audio and/or displayoutput to inform the user of the maneuver that causes the device 100 togo off-route (block 2504). After following the maneuver, the controller124 or mobile application 218 may detect that it the current position ofthe device is at an unmarked location (e.g., a parking lot) and that thedevice 100 is near (e.g., less than X meters from) the destination, andthe controller 124 or mobile application 218 can then initiate arrivalmode (block 2506). In arrival mode, the device 100 may display adistance to the destination and/or an arrow 2505 that points the user tothe destination (block 2508). The device 100 may also provide an audiooutput and/or present an illumination pattern via the indicator lights106 that acts as a bearing pointer (e.g., instead of or in addition tothe arrow 2505) to help point the user to the destination. For example,FIGS. 25B and 25C show map views of the vehicle location 2510 prior tothe maneuver (at block 2504) and after the maneuver (at blocks 2506 and2508) when the device 100 is in arrival mode.

FIGS. 26A and 26B show an example process 2600 for presenting smartnotifications (e.g., message notifications, emails, alerts/warnings,calendar alerts/updates, and so forth) via the device 100 (e.g., via thedisplay 104, speaker 120/audio output interface 144, and/or indicatorlights 106). A smart notification may be received at the mobile device202 and communicated to the device 100 via the mobile application 218(block 2602). For example, the mobile device 202 may receive a textmessage, emails, alert/warning, calendar alert, calendar update, alarmnotification, or the like. The device 100 can determine whether an ASRsession is active and/or device 100 is at or near a route guidancemaneuver (block 2604). The device 100 can wait for the ASR session toend or for the device 100 to clear the route guidance maneuver to avoidinterrupting active ASR or disrupting route guidance near a maneuver(block 2606). When the device 100 determines that there are no activeASR sessions and that the device 100 is not at or near a route guidancemaneuver, the device 100 may provide an audio and/or display output topresent a notification title to the user (block 2608). If the user doesnot provide any instructions or queries for the device 100, the device100 may display the message for some time (block 2610) and then clearthe message from the display 104 (block 2612). If the user instructs thedevice 100 to play the message from server 206/208 (or presses theaction button to cause the device 100 to play the message), the device100 may communicate the instructions to server 206/208 while displayingan illumination pattern (e.g., spinning blue light) to let the user knowthat the device 100 is processing the user's request (block 2614). Whenthe message has been received by the device 100 from server 206/208, thedevice 100 can provide an audio and/or display output including themessage for the user to view or hear (block 2616). If the user instructsthe device 100 to clear the message from server 206/208, the device 100may communicate the instructions to server 206/208 while displaying anillumination pattern (e.g., spinning blue light) to let the user knowthat the device 100 is processing the user's request (block 2620). Whenthe message has been cleared (e.g., deleted/archived), the device 100may provide an audio (e.g., success tone) and/or display output tonotify the user that the request to clear the message has beensuccessfully processed (block 2622).

FIGS. 27A and 27B show an example process 2700 for presenting ADASalerts/warnings (e.g., a forward collision warning) via the device 100(e.g., via the display 104, speaker 120/audio output interface 144,and/or indicator lights 106). While the device 100 is in motion, aforward collision warning event can be detected (block 2702). Forexample, the device 100 can detect when there is less than a thresholdproximity (e.g., less than a threshold distance) to an object in a fieldof view of the camera 122 based on the image(s) and/or video recorded bythe camera 122 while the device 100 is in motion. The device 100 maydetermine if a lane departure warning is active (block 2704). If thereis no lane departure warning, the device 100 can provide a displayoutput, audio output, and/or illumination pattern (e.g., illuminating afront portion of the light ring) to present the forward collisionwarning to the user (block 2708). If there is a lane departure warning,the device 100 can provide a display output, audio output, and/orillumination pattern (e.g., illuminating a front portion of the lightring) to present the forward collision warning to the user in additionto providing a display output, audio output, and/or illumination pattern(e.g., illuminating a side portion of the light ring) to present thelane departure warning to the user (block 2706). As shown in FIG. 27B,if the forward collision warning is detected when route guidance isactive and the device 100 is at or near a maneuver, the device 100 canprovide a display output that presents the maneuver to the user and canprovide an audio output and/or illumination pattern (e.g., illuminatinga front portion of the light ring) to present the forward collisionwarning to the user (block 2710).

FIGS. 28A and 28B show an example process 2800 for presenting ADASalerts/warnings (e.g., a lane departure warning) via the device 100(e.g., via the display 104, speaker 120/audio output interface 144,and/or indicator lights 106). While the device 100 is in motion, a lanedeparture warning event can be detected (block 2802). For example, thedevice 100 can detect when there at least a threshold deviation from areference alignment between lanes in a field of view of the camera 122based on the image(s) and/or video recorded by the camera 122 while thedevice 100 is in motion (e.g., the device 100 detects that the vehiclehas crossed over a solid line). The device 100 may determine thedirection of the lane departure (e.g., did the vehicle cross over asolid line on the right or left?) (block 2804). The device 100 maydetermine if a forward collision warning is active (block 2806/2812). Ifthere is no forward collision warning, the device 100 can provide adisplay output, audio output, and/or illumination pattern (e.g.,illuminating a left or right-side portion of the light ring) to presentthe lane departure warning to the user (block 2808/2816). If there is anactive forward collision warning, the device 100 can provide a displayoutput, audio output, and/or illumination pattern (e.g., illuminating afront portion of the light ring) to present the forward collisionwarning to the user in addition to providing an audio output and/orillumination pattern (e.g., illuminating a left or right-side portion ofthe light ring) to present the lane departure warning to the user (block2810/2814). The forward collision warning may be prioritized over thelane departure warning; accordingly, the device 100 may be configured todisplay the forward collision warning and/or provide audio to notify theuser of the forward collision warning even though a lane departurewarning may be simultaneously active on the device 100. As shown in FIG.28B, if the lane departure warning is detected when route guidance isactive and the device 100 is at or near a maneuver, the device 100 canprovide a display output that presents the maneuver to the user and canprovide an audio output and/or illumination pattern (e.g., illuminatinga side portion of the light ring) to present the lane departure warningto the user (block 2818).

FIG. 29 shows an example process 2900 for presenting ADASalerts/warnings (e.g., a speed limit warning) via the device 100 (e.g.,via the display 104, speaker 120/audio output interface 144, and/orindicator lights 106). The device 100 may acquire speed limitinformation from server 208 and detect when the device 100 is over thespeed limit (block 2902). For example, the device 100 can detect whenthe device 100 is moving at a speed at least a threshold speed above thespeed limit (e.g., at least X mph over the speed limit) based onlocation tracking data (e.g., GPS tracking data) and/or speed sensordata. The device 100 can determine whether an ASR session is activeand/or whether device 100 is at or near a route guidance maneuver (block2904). The device 100 can wait for the ASR session to end or for thedevice 100 to clear the route guidance maneuver to avoid interruptingactive ASR or disrupting route guidance near a maneuver (block 2906).When the device 100 determines that there are no active ASR sessions andthat the device 100 is not at or near a route guidance maneuver, thedevice 100 may determine whether the user has traveled below the speedlimit since the last speed limit warning was provided to the user (block2910). If the device 100 determines that the user traveled below thespeed limit after the last speed limit warning was provided to the user,the device 100 may provide a new speed limit warning by providing anaudio output, display output (e.g., presenting the speed limit) and/orillumination pattern (e.g., presenting a red light ring via theindicator lights 106) to alert the user that he or she is speeding(block 2912). The device 100 can discontinue the display output (andreturn to a previous display, e.g., route guidance) and may alsodiscontinue and/or reduce the intensity of the illumination pattern(e.g., by turning off segments of the light ring or reducing brightness)after a period (e.g., after t seconds) of time (block 2914). The device100 may completely discontinue the speed limit warning when the device100 is traveling below the speed limit or is no longer traveling morethan the threshold speed above the speed limit (block 2916).

FIGS. 30A and 30B show an example process 3000 for interacting with thedevice 100 to record content (e.g., images and/or video footage) via thecamera 122. For example, the device 100 can record images and/or videofootage in a “time-lapse” format that can be shared with contacts and/oron social media applications/websites. After the wakeup word is spokenor the action button is pressed, the device 100 can listen for a spokeninstruction or query from the user. The user may provide a voice input(e.g., a spoken instruction/query) for the device 100 to record atime-lapse (block 3002). In some implementations, the user queriesserver 206 and/or server 208 to provide instructions to the device 100for recording a time-lapse. The device 100 can determine if the camera122 is already being used to record a time-lapse or content in any otherform (block 3004). If the camera 122 is already recording time-lapsecontent, the device 100 may provide an audio output (e.g., notification“beep”) to the user to let the user know that the camera 122 is alreadyrecording content (block 3006). If the camera 122 is not alreadyrecording time-lapse content, the device 100 may assess the memory(e.g., built-in or installed memory) to determine if the memory isoperable and if there is enough free space in memory to store time-lapsecontent (block 3008). If the memory is inoperable or if there is notenough free space in memory to store time-lapse content (e.g., less thanX GB of memory), the device 100 may provide an audio, display, and/orillumination pattern output to notify the user of a memory error (e.g.,to notify the user of inoperable/insufficient memory) (block 3012). Ifthe memory is operable and has enough free space to record time-lapsecontent (e.g., at least X GB of memory), the device 100 may beginrecording images and/or video content via the camera 122 and may providean audio output (e.g., success “beep”) to notify the user that thecamera 122 has begun recording content (block 3010).

As shown in FIG. 30B, the user can stop recording time-lapse content byproviding a voice input/command for the device 100. After the wakeupword is spoken or the action button is pressed, the device 100 canlisten for a spoken instruction or query from the user. The user mayprovide a voice input (e.g., a spoken instruction/query) for the device100 to stop time-lapse recording (block 3014). The device 100 candetermine if the camera 122 is already being used to record a time-lapseor content in any other form (block 3016). If the device 100 isrecording content via the camera 122, the device 100 can stop recordingthe content and may also provide an audio and/or display output tonotify the user that the device 100 has stopped recording content viathe camera 122 (block 3018). If the device 100 is not recording contentvia the camera 122, the device 100 may still provide an audio and/ordisplay output to notify the user that the device 100 has stoppedrecording content via the camera 122 (block 3020).

The device 100 may provide various symbolic and/or textual outputs viathe display 104 to alert the user of maneuvers during route guidance.For example, FIGS. 31A through 31E illustrate examples of symbols thatcan be provided via the display 104 to indicate maneuvers (e.g., turns),traffic information, and other guidance notifications. Inimplementations, the device 100 may be configured to provide symbolicoutputs via the display 104 to enable presentation of information fromthe device 100 to the user with a small viewing area (e.g., a displaythat may be less than one inch wide in some embodiments).

Various configurations of the system 200 including the device 100, themobile application 218 running on the mobile device 202, and the vehicleaudio system 204 are illustrated in FIGS. 32 through 42B. In anembodiment shown in FIGS. 32 and 33, a configuration 3200 is shown toinclude two interfaces for a user 3202. The user 3202 can interface withthe mobile application 218 using the device 100 and a user interface3206 (e.g., a graphical user interface (GUI)) of the mobile application218 that is provided via the display 248 of the mobile device 202. Themobile application 218 can also include a device interface 3204 thatfacilitates connectivity between the mobile application 218 and thedevice 100 (e.g., via communications between short-range transceivers132 and 220). The mobile application 218 may also be configured tocommunicate directly with the vehicle audio system 204 via theshort-range communications transceiver 220 of the mobile device 202 orindirectly by connecting to the vehicle audio system 204 through thedevice 100. For example, the device 100 may send audio signals to thevehicle audio system 204 after receiving the audio from the mobileapplication 218. In embodiments, the mobile application 218 alsoincludes cached text-to-speech (TTS) data 3212 (e.g., cached navigationinstructions) and user settings 3208. The mobile application 218includes a voice services client 3214 for connecting to the voiceservices (VS) server 206 that can provide speech recognizer 3222 andspeech synthesizer 3224 services. The mobile application 218 can alsoinclude an audio decoder 3216 and a music/media player 3210 foroutputting information (e.g., TTS) received via the VS client 3214. Inembodiments, the device 100 can be configured to receive a spokeninstruction or query form a user via the microphone 110 and configuredto send data associated with the spoken instruction or query to themobile application 218 running on the mobile device 202 via theshort-range communications transceiver 132. The mobile application 218can be configured to cause the mobile device 202 to transmit an audiooutput based on the spoken instruction or query via the short-rangecommunications transceiver 220 to the vehicle audio system 204 or to thedevice 100 so that the device 100 can output the audio via the speaker120 or the audio output interface 144. In embodiments, the mobileapplication 218 is configured to cause the mobile device 202 to transmitthe data associated with the spoken instruction or query to server 206via the cellular transceiver 222 for voice service processing and isfurther configured to cause the mobile device 202 to receive informationfor the audio output from the server 206 in response to the dataassociated with the spoken instruction or query.

The mobile application 218 further includes a navigation controller 3220and navigation interface 3218 for connecting to the navigation/mappingservices (NAV) server 208 that can provide navigation services 3228 andstore user data 3226 (e.g., user profiles for route guidance and otherlocation based services). In implementations, the VS server 206 and theNAV server 208 can be connected by a navigation skill 3822 service thatcan be enabled (e.g., via the mobile application 218) to provide accessto the NAV server 208 through the VS server 206. For example, when thenavigation skill 3822 is enabled, the mobile application 218 can beconfigured to cause the mobile device 202 to transmit the dataassociated with the spoken instruction or query to server 206 for voiceservice processing; receive navigation command data from server 206based on the spoken instruction or query; transmit the navigationcommand data and data associated with the current position of the device100 to server 208 via the cellular transceiver for navigation serviceprocessing; and receive information for an audio/display output (e.g.,navigation data/instructions) from server 208 in response to thenavigation command data and the data associated with the currentposition of the device 100. In another implementation, the mobileapplication 218 can be configured to cause the mobile device 202 totransmit data associated with the current position of the device 100 anddata associated with the spoken instruction or query to server 206 forvoice service processing; receive navigation command data from server206 based on the data associated with the position of the device 100 andthe spoken instruction or query; transmit the navigation command data toserver 208 via the cellular transceiver for navigation serviceprocessing; and receive information for an audio/display output (e.g.,navigation data/instructions) from server 208 in response to thenavigation command data.

As shown in FIG. 33, the mobile application 218 can further include astreaming audio client 3334 that facilitates connectivity between themobile application 218 and media services 3332 (e.g., PANDORA, SPOTIFY,AUDIBLE, and the like). The user interface 3306 may include displaycomponents 3348 that show information associated with media content(e.g., music) received from the media services 3332 and/or mediacontrols 3350. The device 100 can also be configured to control mediaplayback through the mobile application 218 based on voice inputs (e.g.,spoke instructions/queries) received from the user. The mobileapplication 218 may also include additional components 3352 that mayprovide interconnectivity among the various clients/interfaces of themobile application 218 and/or additional clients 3354 that providevarious alerts (e.g., broadcast alerts, mobile device 202 alerts, etc.).In embodiments, the mobile application 218 can include an audioprocessing module 3342 configured to decode and, in some cases, merge aplurality of audio streams (e.g., audio streaming, route guidance, ASRresponses, etc.) that may be output via A2DP 3346 (Bluetooth streaming)and/or another audio output interface to the vehicle audio system 204,or to the device speakers 120 or mobile device speakers 246.

In embodiments, the device 100 can be configured to allow for system 200configurations based on capabilities of the mobile device 202 and thevehicle audio system 204, and/or optional configurations that are basedon user selection. FIG. 34 shows an optional configuration 3400 whereaudio outputs (e.g., route guidance, media streaming, ASR responses,hands-free calls, etc.) are transmitted through the vehicle audio system204 by the mobile device 202. For example, the mobile device 202 may beconfigured to send and receive data to and from the vehicle audio system204 based on a communication channel (e.g., A2DP channel) establishedbetween short-range communications (e.g., Bluetooth) transceivers 220and 262. FIG. 35 illustrates another optional configuration 3500 whereaudio outputs (e.g., route guidance, media streaming, ASR responses,hands-free calls, etc.) are transmitted through the vehicle audio system204 by the device 100. For example, the mobile device 202 may beconfigured to send and receive data to and from the device 100 based ona communication channel (e.g., A2DP channel) established betweenshort-range communications (e.g., Bluetooth) transceivers 132 and 220.The device 100 can then transmit audio outputs associated with the datathrough an audio output interface 144 (e.g., audio line out connection)to the vehicle audio system 204 (e.g., through an auxiliary input/linein connection).

FIG. 36 illustrates an example configuration 3600 of the device 100employing TTS for route guidance. For example, the device 100 can beconfigured to provide route guidance (e.g., display and/or audio)outputs for a user 3602 based on cached data 3604 (e.g., cached TTSdata). The device 100 can be configured to connect to a TTS engine 3606to download additional TTS data for upcoming route guidance instructionswhich are then stored in the cached data 3604.

FIG. 37 illustrates an example configuration 3700 of the device 100employing data from NAV server 208 and TTS for route guidance. Forexample, the device 100 can be configured to provide route guidance(e.g., display and/or audio) outputs for a user 3702 based on cacheddata 3704 (e.g., cached TTS data). The controller 124 or mobileapplication 218 can be configured to connect to a TTS engine 3706 todownload additional TTS data for upcoming route guidance instructionswhich are then stored in the cached data 3704. The controller 124 ormobile application 218 can also be configured to connect to anavigation/mapping engine 3712 and a user database 3710, which may bothbe available through server 208, to receive additional navigation data(e.g., route guidance instructions) or other location based information.In some embodiments, the device 100 is configured to connect to the TTSengine 3706, navigation/mapping engine 3712, and/or user database 3710through a back-channel endpoint 3708.

FIG. 38 illustrates an example configuration 3800 of the device 100employing data from VS server 206 for media applications. For example,the controller 124 or mobile application 218 can be configured toprovide media (e.g., streaming audio, news, etc.) for a user 3802 basedon voice inputs (e.g., spoken instructions/queries) received from theuser 3802. The controller 124 or mobile application 218 can beconfigured to connect to VS server 206 to process the voice inputs andcan connect to remote media sources 3808 (e.g., SPOTIFY, PANDORA, etc.)to receive information for the user 3802 based on the user's voiceinputs. The controller 124 or mobile application 218 can also beconfigured to connect to a user database 3806, which may be availablethrough server 206 or server 208. In some embodiments, the controller124 or mobile application 218 is configured to connect to the userdatabase 3806 through a back-channel endpoint 3804.

FIG. 39 illustrates an example configuration 3900 of the controller 124or mobile application 218 employing data from VS server 206 and NAVserver 208 to search for a location and/or calculate navigation (e.g.,route guidance) data. For example, the controller 124 or mobileapplication 218 can be configured to provide search locations and/orroute guidance (e.g., display and/or audio) outputs for a user 3902based on voice inputs (e.g., spoken instructions/queries) received fromthe user 3902. The controller 124 or mobile application 218 can beconfigured to connect to VS server 206 to process the voice inputs andcan connect to a navigation/mapping engine 3910 and a phrase generator3912, which may both be available through server 208, to receive searchresults, navigation data (e.g., route guidance instructions, timing,etc.) or other location based information. A navigation skill 3908 canbe enabled through the mobile application 218 to facilitate connectivitybetween the VS server 206 and NAV server 208. The controller 124 ormobile application 218 can also be configured to connect to a userdatabase 3906, which may be available through server 206 or server 208.In some embodiments, the device 100 is configured to connect to the userdatabase 3906 through a back-channel endpoint 3904.

FIG. 40 illustrates an example configuration 4000 of the controller 124or mobile application 218 employing data from VS server 206 and NAVserver 208 to search for one or more locations and/or calculatenavigation (e.g., route guidance) data. For example, the device 100 canbe configured to provide search locations and/or route guidance (e.g.,display and/or audio) outputs for a user 4002 based on voice inputs(e.g., spoken instructions/queries) received from the user 4002. Thecontroller 124 or mobile application 218 can be configured to connect toVS server 206 to process the voice inputs and can connect to anavigation/mapping engine 4010 and a phrase generator 4012, which mayboth be available through server 208, to receive search results,navigation data (e.g., route guidance instructions, timing, etc.) orother location based information. A navigation skill 4008 can be enabledthrough the mobile application 218 to facilitate connectivity betweenthe VS server 206 and NAV server 208. The controller 124 or mobileapplication 218 can also be configured to connect to a user database4006, which may be available through server 206 or server 208. In someembodiments, the controller 124 or mobile application 218 is configuredto connect to the user database 4006 through a back-channel endpoint4004. As shown in FIG. 40, the controller 124 or mobile application 218can provide audio outputs indicating a plurality of suggestions (e.g.,search results) based on the user's query, where the user 4002 canrespond by accepting or rejecting suggestions provided by the device100.

FIGS. 41A and 41B illustrate an example configuration 4100 of the device100 employing data from VS server 206 and NAV server 208 to providelocation based information (e.g., traffic information/updates) duringroute guidance. For example, the device 100 can be configured to providedisplay and/or audio outputs associated with traffic information for auser 4102 based on voice inputs (e.g., spoken instructions/queries)received from the user 4102. The controller 124 and/or mobileapplication 218 can be configured to connect to a TTS engine 4110 todownload additional TTS data for upcoming route guidance instructionsand/or traffic information. The controller 124 and/or mobile application218 can also be configured to connect to VS server 206 to process thevoice inputs and can connect to a navigation/mapping engine 4112,traffic provider 4114, and a phrase generator 4118, which may both beavailable through server 208, to receive search results, navigation data(e.g., route guidance instructions, timing, etc.) or other locationbased information (e.g., traffic information/updates). A navigationskill 4108 can be enabled through the mobile application 218 tofacilitate connectivity between the VS server 206 and NAV server 208.The controller 124 and/or mobile application 218 can also be configuredto connect to a user database 4106, which may be available throughserver 206 or server 208. In some embodiments, the device 100 isconfigured to connect to the user database 4106, TTS engine 4110,navigation/mapping engine 4112, traffic provider 4114, and/or phrasegenerator 4118, through a back-channel endpoint 4104. As shown in FIG.41A, the device 100 can provide audio outputs associated with trafficinformation while providing route guidance (e.g., navigation data) forthe user 4102. The controller 124 and/or mobile application 218 can alsobe configured to provide alternative routes for the user 4102 andreceive voice inputs from the user 4102 to change the route based on thetraffic information.

FIGS. 42A and 42B illustrate an example configuration 4200 of the device100 employing data from VS server 206 and NAV server 208 to search forone or more locations and/or calculate navigation (e.g., route guidance)data, where a home mobile application interface device 4206 (e.g., anAMAZON ECHO device or the like) is used to receive voice inputs (e.g.,spoken instructions/queries) from a user 4202. For example, the device100 can be configured to provide search locations and/or route guidance(e.g., display and/or audio) outputs for a user 4202 based on voiceinputs (e.g., spoken instructions/queries) received from the user 4202via the home mobile application interface device 4206. The device 100and the home mobile application interface device 4206 can be configuredto connect to VS server 206 to process the voice inputs and can connectto a navigation/mapping engine 4218 and a phrase generator 4216, whichmay both be available through server 208, to receive search results,navigation data (e.g., route guidance instructions, timing, etc.), orother location based information. A navigation skill 4214 can be enabledthrough the mobile application 218 to facilitate connectivity betweenthe VS server 206 and NAV server 208. The device 100 and the home mobileapplication interface device 4206 can also be configured to connect to auser database 4210, which may be available through server 206 or server208. In some embodiments, the device 100 and the home mobile applicationinterface device 4206 are configured to connect to the user database4210, TTS engine 4212, navigation/mapping engine 4218, and/or phrasegenerator 4216, through a back-channel endpoint 4208. As shown in FIG.42A, the device 100 may be turned off, and yet the user 4202 caninteract with the home mobile application interface device 4206 that isconfigured to provide audio outputs indicating a plurality ofsuggestions (e.g., search results) based on the user's query, where theuser 4202 can respond by accepting or rejecting suggestions andinstructing the home mobile application interface device 4206 to savethe results for route guidance to be provided by the device 100. Whenthe device 100 is on, the device 100 can be configured to initiate routeguidance and/or ask the user 4202 whether to initiate route guidance toa saved search result.

Although the technology has been described with reference to theembodiments illustrated in the attached drawing figures, equivalents maybe employed and substitutions made herein without departing from thescope of the technology as recited in the claims. For example, thecomponents described herein need not be physically connected to oneanother since wireless communication among the various depictedcomponents is permissible and intended to fall within the scope of thepresent invention. Components illustrated and described herein aremerely examples of a device and components that may be used to implementthe embodiments of the present invention and may be replaced with otherdevices and components without departing from the scope of theinvention.

What is claimed is:
 1. A device for interfacing with a mobileapplication running on a mobile device to provide vehicle navigationassistance, the device comprising: a microphone; a location determiningcomponent; a display; a plurality of indicator lights; a short-rangecommunications transceiver configured to send and receive communicationsto and from the mobile device; and a controller communicatively coupledwith the microphone, the location determining component, the display,the plurality of indicator lights, and the short-range communicationstransceiver, the controller configured to: receive a spoken instructionor query from a user via the microphone; send data associated with thespoken instruction or query to the mobile application running on themobile device via the short-range communications transceiver; receivenavigation data based on the spoken instruction or query from the mobileapplication running on the mobile device via the short-rangecommunications transceiver; determine a current position based on one ormore signals received via the location determining component; andprovide at least one of a symbolic output or a textual output via thedisplay based on the navigation data and the current position.
 2. Thedevice of claim 1, wherein the at least one of the symbolic output orthe textual output is associated with a next maneuver for the user tofollow based on the navigation data and the current position, andwherein the controller is further configured to cause the plurality ofindicator lights to display a dynamic illumination pattern based on adistance between the current position and the next maneuver.
 3. Thedevice of claim 1, wherein the at least one of the symbolic output orthe textual output is associated with a distance to a destination basedon the navigation data and the current position.
 4. The device of claim3, wherein the controller is configured to initiate an arrival mode whenthe current position is less than a threshold distance from thedestination or the current position corresponds to an unmarked location,and wherein the at least one of the symbolic output or the textualoutput comprises an arrow pointing towards the destination and adistance to the destination.
 5. The device of claim 3, wherein thecontroller is configured to cause the plurality of indicator lights todisplay a dynamic illumination pattern based on the distance between thecurrent position and the destination.
 6. The device of claim 1, furthercomprising: a camera communicatively coupled to the controller, whereinthe controller is further configured to receive at least one image orvideo recorded by the camera while the device is in motion and furtherconfigured to cause the plurality of indicator lights to display aforward collision warning illumination pattern in response to detectingless than a threshold proximity to an object in a field of view of thecamera based on the at least one image or video recorded by the camerawhile the device is in motion.
 7. The device of claim 1, furthercomprising: a camera communicatively coupled to the controller, whereinthe controller is further configured to receive at least one image orvideo recorded by the camera while the device is in motion and furtherconfigured to cause the plurality of indicator lights to display a lanedeparture warning illumination pattern in response to detecting at leasta threshold deviation from a reference alignment between lanes in afield of view of the camera based on the at least one image or videorecorded by the camera while the device is in motion.
 8. A system forvehicle navigation assistance, the system comprising: a mobile deviceincluding a memory and a processor configured to run a mobileapplication from the memory; and device for interfacing with the mobileapplication running on the mobile device, the device including: amicrophone; a location determining component; a display; a plurality ofindicator lights; a short-range communications transceiver configured tosend and receive communications to and from the mobile device; and acontroller communicatively coupled with the microphone, the locationdetermining component, the display, the plurality of indicator lights,and the short-range communications transceiver, the controllerconfigured to: receive a spoken instruction or query from a user via themicrophone; send data associated with the spoken instruction or query tothe mobile application running on the mobile device via the short-rangecommunications transceiver; receive navigation data based on the spokeninstruction or query from the mobile application running on the mobiledevice via the short-range communications transceiver; determine acurrent position based on one or more signals received via the locationdetermining component; and provide at least one of a symbolic output ora textual output via the display based on the navigation data and thecurrent position.
 9. The system of claim 8, wherein the at least one ofthe symbolic output or the textual output is associated with a nextmaneuver for the user to follow based on the navigation data and thecurrent position, and wherein the controller is further configured tocause the plurality of indicator lights to display a dynamicillumination pattern based on a distance between the current positionand the next maneuver.
 10. The system of claim 8, wherein the at leastone of the symbolic output or the textual output is associated with adistance to a destination based on the navigation data and the currentposition.
 11. The system of claim 10, wherein the controller isconfigured to initiate an arrival mode when the current position is lessthan a threshold distance from the destination or the current positioncorresponds to an unmarked location, and wherein the at least one of thesymbolic output or the textual output comprises an arrow pointingtowards the destination and a distance to the destination.
 12. Thesystem of claim 10, wherein the controller is configured to cause theplurality of indicator lights to display a dynamic illumination patternbased on the distance between the current position and the destination.13. The system of claim 8, wherein the device further includes: a cameracommunicatively coupled to the controller, wherein the controller isfurther configured to receive at least one image or video recorded bythe camera while the device is in motion and further configured to causethe plurality of indicator lights to display a forward collision warningillumination pattern in response to detecting less than a thresholdproximity to an object in a field of view of the camera based on the atleast one image or video recorded by the camera while the device is inmotion.
 14. The system of claim 8, wherein the device further includes:a camera communicatively coupled to the controller, wherein thecontroller is further configured to receive at least one image or videorecorded by the camera while the device is in motion and furtherconfigured to cause the plurality of indicator lights to display a lanedeparture warning illumination pattern in response to detecting at leasta threshold deviation from a reference alignment between lanes in afield of view of the camera based on the at least one image or videorecorded by the camera while the device is in motion.
 15. A method ofproviding vehicle navigation assistance, the method comprising: runninga mobile application for vehicle navigation assistance on a mobiledevice; pairing a mobile application interface device with the mobiledevice, the mobile application interface device having a microphone, alocation determining component, a display, a plurality of indicatorlights, and a short-range communications transceiver operable to sendand receive communications to and from the mobile device; receiving aspoken instruction or query from a user via the microphone of the mobileapplication interface device; sending data associated with the spokeninstruction or query to the mobile application running on the mobiledevice via the short-range communications transceiver of the mobileapplication interface device; receiving navigation data based on thespoken instruction or query from the mobile application running on themobile device via the short-range communications transceiver of themobile application interface device; determining a current positionbased on one or more signals received via the location determiningcomponent of the mobile application interface device; and providing atleast one of a symbolic output or a textual output via the display ofthe mobile application interface device based on the navigation data andthe current position, wherein the at least one of the symbolic output orthe textual output is associated with at least one of a next maneuverfor the user to follow or a distance to a destination.
 16. The method ofclaim 15, further comprising: causing the plurality of indicator lightsto display a dynamic illumination pattern based on a distance betweenthe current position and the next maneuver.
 17. The method of claim 15,further comprising: causing the plurality of indicator lights to displaya dynamic illumination pattern based on the distance between the currentposition and the destination.
 18. The method of claim 15, furthercomprising: initiating an arrival mode when the current position is lessthan a threshold distance from the destination or the current positioncorresponds to an unmarked location, wherein the at least one of thesymbolic output or the textual output comprises an arrow pointingtowards the destination and a distance to the destination.
 19. Themethod of claim 15, further comprising: recording at least one image orvideo with a camera of the mobile application interface device while themobile application interface device is in motion; and causing theplurality of indicator lights to display a forward collision warningillumination pattern in response to detecting less than a thresholdproximity to an object in a field of view of the camera based on the atleast one image or video recorded by the camera while the mobileapplication interface device is in motion.
 20. The method of claim 15,further comprising: recording at least one image or video with a cameraof the mobile application interface device while the mobile applicationinterface device is in motion; and causing the plurality of indicatorlights to display a lane departure warning illumination pattern inresponse to detecting at least a threshold deviation from a referencealignment between lanes in a field of view of the camera based on the atleast one image or video recorded by the camera while the mobileapplication interface device is in motion.